Phase 2: the event core
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s

Implements PLAN.md Phase 2: the library every later phase is built on,
as pure code under server/internal/matrix with no HTTP and no storage.

canonicaljson: the canonical JSON encoding, hand-written because
encoding/json escapes <, > and & and accepts floats. Integers only, in
[-(2^53)+1, 2^53-1]; lone surrogate escapes rejected rather than turned
into U+FFFD.

signing: the Signing JSON algorithm alongside the existing key handling,
and one lenient base64 decoder shared by the packages above it.

id: identifier grammars as they appear inside events, deliberately laxer
than userapi's account rules because rooms carry historical user IDs.

roomversion: the registry. Versions 3 to 12 are supported, each written
out in full as a Rules value with the flags the other packages read.
Versions 1 and 2 are known but refused: their event format and state
resolution v1 wait for federation. createRoom will default to 11, as the
spec asks for a while after 1.16, with 12 available.

event: PDU parsing and validation, content and reference hashes, event
IDs, the redaction algorithm for every supported version, signing and
verification, and a Builder.

auth: the authorisation rules in spec order, power levels with their
defaults, v12 creators with infinite power, the auth events selection
algorithm, rule 3's auth_events checks, and the separate test for
whether a redaction applies. Comments use the room version 12 rule
numbering, with the version 11 numbers given where they differ.

stateres: state resolution v2, and version 12's amendment (an empty
starting state and the conflicted state subgraph).

Tested against the spec's own vectors byte for byte: the canonical JSON
examples, both JSON signing vectors and both event signing vectors.
State resolution ports scenarios from Synapse's suite. canonicaljson
and event.Parse have fuzz targets.

The capabilities endpoint now advertises the registry instead of a
hard-coded "11". docs/event-core.md describes the packages and the
known gaps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
tajniak81andClaude Opus 5.5 committed 2026-10-15 22:23:35 +02:00
1 parent 6004bb01de
commit 2fbd5f53ce
30 files changed
+8928 -9

No files matched your search

+20 -6
View File
@@ -4,15 +4,29 @@ import (
"net/http"
"github.com/dariuszjaniak/warp/server/internal/httputil"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
"github.com/dariuszjaniak/warp/server/internal/userapi"
)
// DefaultRoomVersion is what this server creates rooms at.
// DefaultRoomVersion is what this server creates rooms at. The room
// version registry decides which one that is, and why.
const DefaultRoomVersion = string(roomversion.Default)
// availableRoomVersions is the m.room_versions.available map.
//
// PLAN.md's decision is to implement the latest stable room version first
// and backfill older ones only as federation needs them, so the registry
// here has exactly one entry until the event core lands in Phase 2.
const DefaultRoomVersion = "11"
// Every version Warp implements is marked stable, because every version
// the spec still lists is stable — the "unstable" marking is for versions
// under development, which we do not implement. Versions 1 and 2 are in
// the registry but not supported, and so are left out: a client offered a
// version it cannot be given would get an error on createRoom.
func availableRoomVersions() map[string]string {
supported := roomversion.Supported()
out := make(map[string]string, len(supported))
for _, v := range supported {
out[string(v)] = "stable"
}
return out
}
// roomVersionsCapability is the m.room_versions capability.
type roomVersionsCapability struct {
@@ -52,7 +66,7 @@ func (c *ClientAPI) handleCapabilities(w http.ResponseWriter, _ *http.Request, _
Capabilities: capabilities{
RoomVersions: roomVersionsCapability{
Default: DefaultRoomVersion,
Available: map[string]string{DefaultRoomVersion: "stable"},
Available: availableRoomVersions(),
},
ChangePassword: booleanCapability{Enabled: false},
SetDisplayName: booleanCapability{Enabled: true},
+559
View File
@@ -0,0 +1,559 @@
// Package auth implements the Matrix authorisation rules: given an event
// and the room state it refers to, may that event be in the room?
//
// This is the part of a homeserver that must not be improvised. Every
// server in a room runs these rules over the same events and has to reach
// the same verdict, or the room's history forks. The rules are therefore
// written out here in the order and the shape the spec gives them, with
// the room version differences driven by the roomversion registry rather
// than by guesswork, and the comments say which rule is which so that the
// code can be read against the spec side by side.
//
// # Rule numbers
//
// "Rule N" in these comments means the numbering on the room version 12
// page, the most recent one. Version 12 inserted a new rule 2 (the room ID
// is the create event's ID), so on the pages for versions 6 to 11 each rule
// after the first is one lower: rule 5 here is rule 4 there, rule 10 here
// is rule 9 there. Versions 1 to 5 happen to line up with version 12 again
// from rule 5 on, because their rule 4 was the m.room.aliases rule that
// version 6 removed. Where a sub-rule exists in only some versions, or is
// numbered differently beyond that shift, the comment says so.
//
// Nothing here touches storage or the network. The one exception is the
// restricted join rule, which needs somebody else's signature verified;
// that comes in through the Verifier interface.
//
// What this package does *not* decide:
//
// - Whether a redaction should be applied. The rules allow nearly every
// m.room.redaction event; whether the redaction takes effect is a
// separate test (ShouldApplyRedaction) the roomserver runs once it has
// both events.
// - Soft failure. An event can be allowed by the rules against its own
// auth events and still be refused a place in the room's forward
// extremities because it fails against the *current* state. That is
// the roomserver's business (PLAN.md phase 3).
package auth
import (
"context"
"encoding/json"
"errors"
"fmt"
"slices"
"github.com/dariuszjaniak/warp/server/internal/matrix/event"
"github.com/dariuszjaniak/warp/server/internal/matrix/id"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
)
// The join rules a room may have.
const (
JoinRulePublic = "public"
JoinRuleInvite = "invite"
JoinRuleKnock = "knock"
JoinRuleRestricted = "restricted"
JoinRuleKnockRestricted = "knock_restricted"
JoinRulePrivate = "private"
)
// ErrRejected means the authorisation rules refused the event. Every
// rejection from Check wraps it, and the message says which rule said no.
var ErrRejected = errors.New("rejected by the authorisation rules")
// ErrNoVerifier means a decision needed somebody else's signature checked
// and no Verifier was supplied. It is kept separate from ErrRejected
// because the event is not known to be bad — we are simply not equipped to
// judge it, and retrying with a verifier may well allow it.
var ErrNoVerifier = errors.New("cannot verify the signature this rule needs")
// Verifier checks that an event carries a valid signature from a server.
//
// Only one rule needs this: a join under the restricted join rule is
// authorised by a third party, named in
// content.join_authorised_via_users_server, and the event must be signed
// by *that user's* server rather than only by the joining user's. Verifying
// it may mean fetching another server's keys, which is why it is an
// interface here instead of a function.
type Verifier interface {
VerifyEventSignature(ctx context.Context, serverName string, e *event.Event) error
}
// reject builds a rejection error.
func reject(format string, args ...any) error {
return fmt.Errorf("%w: %s", ErrRejected, fmt.Sprintf(format, args...))
}
// Check applies the authorisation rules to an event.
//
// st is the room state the event is judged against: for an event arriving
// from a client or over federation that is the state its auth_events name,
// and during state resolution it is the state resolved so far. A nil error
// means the event is allowed.
//
// verifier may be nil. It is needed only for a join authorised by another
// user under the restricted join rule; without one, such a join fails with
// ErrNoVerifier rather than being waved through.
func Check(ctx context.Context, e *event.Event, st State, verifier Verifier) error {
rules := e.Rules()
// Rule 1: the create event answers to nothing but itself.
if e.Type() == event.TypeCreate {
return checkCreate(e)
}
create := st.Create()
if create == nil {
// Not one of the numbered rules but a precondition for all of them:
// without the create event we do not know the room's creators, and
// from room version 12 we do not even know its ID. (The numbered
// rule about a missing create event, 2.4 on the version 11 page,
// judges the declared auth_events and lives in CheckAuthEvents.)
return reject("the room state has no m.room.create event")
}
// Rule 2 (room version 12+): the room ID is the create event's ID, so
// an event claiming a different room is not in this room.
if rules.RoomIDIsCreateEventID {
roomID, err := event.RoomIDOfCreateEvent(create)
if err != nil {
return err
}
if e.RoomID() != roomID {
return reject("room_id %q is not the ID of this room's m.room.create event (%q)", e.RoomID(), roomID)
}
}
// Rule 4: a room created with m.federate false never leaves the
// creator's server.
if err := checkFederate(e, create); err != nil {
return err
}
pl, err := ResolvePowerLevels(st.PowerLevels(), create, rules)
if err != nil {
return err
}
// The m.room.aliases rule, rule 4 on the pages for room versions 1 to
// 5 and absent from version 6 on: every server managed its own alias
// list, so an aliases event needed only to match its sender's domain.
// It was dropped because it let any server in the room edit state.
if rules.AliasesEventAuth && e.Type() == event.TypeAliases {
return checkAliases(e)
}
// Rule 5: membership changes have their own rules, and are the only
// thing a user who is not in the room may send.
if e.Type() == event.TypeMember {
return checkMember(ctx, e, st, create, pl, verifier)
}
// Rule 6: everything else requires the sender to be in the room.
if m := st.Membership(e.Sender()); m != event.MembershipJoin {
return reject("the sender is not joined to the room (membership is %q)", m)
}
senderLevel := pl.UserLevel(e.Sender())
// Rule 7: an invite to a third party needs the power to invite.
if e.Type() == event.TypeThirdPartyInvite {
if senderLevel < pl.Invite {
return reject("the sender's power level %s is below the invite level %d",
levelString(senderLevel), pl.Invite)
}
return nil
}
// Rule 8: the sender must be allowed to send this type of event.
if required := pl.SendLevel(e.Type(), e.IsState()); senderLevel < required {
return reject("the sender's power level %s is below the %d needed to send %s",
levelString(senderLevel), required, e.Type())
}
// Rule 9: state keyed by a user ID belongs to that user. It is what
// stops one member rewriting another's per-user state.
if sk, ok := e.StateKey(); ok && len(sk) > 0 && sk[0] == '@' && sk != e.Sender() {
return reject("the state_key %q is a user ID that is not the sender", sk)
}
// Rule 10: a power levels event may not grant or revoke more than the
// sender holds.
if e.Type() == event.TypePowerLevels {
return checkPowerLevels(e, st.PowerLevels(), create, senderLevel)
}
// Rule 11: otherwise, allow.
return nil
}
// checkCreate applies rule 1, the rules for the event that starts a room.
func checkCreate(e *event.Event) error {
rules := e.Rules()
// Rule 1.1: a create event is the root of the DAG.
if len(e.PrevEventIDs()) > 0 {
return reject("an m.room.create event must have no prev_events")
}
// Rule 1.2.
if rules.RoomIDIsCreateEventID {
if e.RoomID() != "" {
return reject("an m.room.create event in a version %s room must not have a room_id", rules.Version)
}
} else {
roomServer, err := id.ServerNameOfRoomID(e.RoomID())
if err != nil {
return reject("room_id: %v", err)
}
senderServer, err := e.SenderServer()
if err != nil {
return reject("sender: %v", err)
}
if roomServer != senderServer {
return reject("the room_id domain %q does not match the sender's domain %q", roomServer, senderServer)
}
}
var content struct {
RoomVersion *string `json:"room_version"`
Creator *string `json:"creator"`
Additional *json.RawMessage `json:"additional_creators"`
}
if err := json.Unmarshal(e.Content(), &content); err != nil {
return reject("m.room.create content: %v", err)
}
// Rule 1.3: a room version we do not implement is one we must not
// pretend to be in.
if content.RoomVersion != nil {
if !roomversion.IsSupported(roomversion.Version(*content.RoomVersion)) {
return reject("room version %q is not one this server recognises", *content.RoomVersion)
}
}
// Rule 1.4 (room version 12+): additional creators must be user IDs.
// Creators cannot be changed later, so a malformed entry here would be
// permanent.
if rules.AdditionalCreators && content.Additional != nil {
var additional []string
if err := json.Unmarshal(*content.Additional, &additional); err != nil {
return reject("m.room.create additional_creators must be an array of strings: %v", err)
}
for _, userID := range additional {
if err := id.ValidateUserID(userID); err != nil {
return reject("m.room.create additional_creators: %v", err)
}
}
}
// Room versions up to 10 required the creator property, which only
// ever repeated the sender.
if rules.CreateEventRequiresCreator && content.Creator == nil {
return reject("an m.room.create event in a version %s room must have a creator", rules.Version)
}
return nil
}
// checkFederate applies rule 4: a room whose create event says
// m.federate false accepts events only from the creator's own server.
func checkFederate(e *event.Event, create *event.Event) error {
var content struct {
Federate *bool `json:"m.federate"`
}
if err := json.Unmarshal(create.Content(), &content); err != nil {
return reject("m.room.create content: %v", err)
}
if content.Federate == nil || *content.Federate {
return nil
}
senderServer, err := e.SenderServer()
if err != nil {
return reject("sender: %v", err)
}
createServer, err := create.SenderServer()
if err != nil {
return reject("the create event's sender: %v", err)
}
if senderServer != createServer {
return reject("the room does not federate and %q is not %q", senderServer, createServer)
}
return nil
}
// checkAliases applies the room version 1 to 5 rule for m.room.aliases.
func checkAliases(e *event.Event) error {
sk, ok := e.StateKey()
if !ok {
return reject("an m.room.aliases event must have a state_key")
}
senderServer, err := e.SenderServer()
if err != nil {
return reject("sender: %v", err)
}
if sk != senderServer {
return reject("the state_key %q is not the sender's server %q", sk, senderServer)
}
return nil
}
// checkPowerLevels applies rule 10: the content is well-formed, and no
// level is moved past the sender's own.
//
// The asymmetry in the rules is deliberate. You may lower a level you
// could have reached yourself, and you may raise one up to your own level,
// but you may not touch anybody whose level equals yours — otherwise two
// administrators could demote each other, and whoever's event won state
// resolution would end up alone in charge.
func checkPowerLevels(e, old, create *event.Event, senderLevel int64) error {
rules := e.Rules()
// Rules 10.1 to 10.3: parsing the content is the check. ResolvePowerLevels
// rejects a level that is not an integer, an events or notifications
// map whose values are not integers, and a users key that is not a
// user ID.
if _, err := ResolvePowerLevels(e, create, rules); err != nil {
return err
}
newContent, err := levelContent(e)
if err != nil {
return err
}
// Rule 10.4 (room version 12+): creators have infinite power, so
// naming one here could only ever be an attempt to take it away.
//
// Being new, this sub-rule shifts the ones after it: on the version 11
// page both halves of their numbers are one lower, so 10.5 here is 9.4
// there and 10.10 here is 9.9.
if rules.CreatorsHaveInfinitePower {
creators, err := Creators(create)
if err != nil {
return err
}
for _, creator := range creators {
if _, ok := newContent.sub("users")[creator]; ok {
return reject("m.room.power_levels must not give a level to the room creator %q", creator)
}
}
}
// Rule 10.5: the first power levels event in a room is always allowed.
// Until it exists the creator is the only person with any power, so
// there is nobody to protect anything from.
if old == nil {
return nil
}
oldContent, err := levelContent(old)
if err != nil {
return err
}
// Rule 10.6: the named levels.
for _, name := range []string{
"users_default", "events_default", "state_default",
"ban", "redact", "kick", "invite",
} {
if err := checkLevelChange(name, "", oldContent.top, newContent.top, senderLevel, false); err != nil {
return err
}
}
// Rules 10.7 and 10.8: the events and notifications maps.
for _, section := range []string{"events", "notifications"} {
oldSection, newSection := oldContent.sub(section), newContent.sub(section)
for _, key := range unionKeys(oldSection, newSection) {
if err := checkLevelChange(key, section, oldSection, newSection, senderLevel, false); err != nil {
return err
}
}
}
// Rules 10.9 and 10.10: the users map, where an entry other than the
// sender's own is protected at the sender's own level rather than
// above it.
oldUsers, newUsers := oldContent.sub("users"), newContent.sub("users")
for _, userID := range unionKeys(oldUsers, newUsers) {
if err := checkLevelChange(userID, "users", oldUsers, newUsers, senderLevel, userID != e.Sender()); err != nil {
return err
}
}
return nil
}
// checkLevelChange compares one level before and after.
//
// A level that is absent on one side is not compared on that side: the
// rules speak of "the current value" and "the new value", and where there
// is no value there is nothing to check.
func checkLevelChange(key, section string, old, new map[string]int64, senderLevel int64, protectPeers bool) error {
oldLevel, hadOld := old[key]
newLevel, hasNew := new[key]
if hadOld && hasNew && oldLevel == newLevel {
return nil
}
where := key
if section != "" {
where = section + "." + key
}
// The peer rule: you may not change or remove the entry of somebody at
// your own level.
if protectPeers && hadOld && oldLevel == senderLevel {
return reject("%s is at the sender's own power level %s and may not be changed by them",
where, levelString(senderLevel))
}
if hadOld && oldLevel > senderLevel {
return reject("%s is %d, above the sender's power level %s", where, oldLevel, levelString(senderLevel))
}
if hasNew && newLevel > senderLevel {
return reject("%s would become %d, above the sender's power level %s", where, newLevel, levelString(senderLevel))
}
return nil
}
// levels holds a power levels event's content as plain integers, keeping
// track of which keys were actually present.
type levels struct {
top map[string]int64
subs map[string]map[string]int64
}
func (l levels) sub(name string) map[string]int64 {
if m, ok := l.subs[name]; ok {
return m
}
return map[string]int64{}
}
// levelContent reads a power levels event's content for comparison.
//
// It parses leniently on purpose: by the time this runs, the new event has
// already been through ResolvePowerLevels, and the old one was accepted
// into the room's state under whatever rules applied then. A value we
// cannot read is left out rather than treated as zero, which would invent
// a level nobody set.
func levelContent(e *event.Event) (levels, error) {
out := levels{top: map[string]int64{}, subs: map[string]map[string]int64{}}
var obj map[string]json.RawMessage
if err := json.Unmarshal(e.Content(), &obj); err != nil {
return out, reject("m.room.power_levels content: %v", err)
}
rules := e.Rules()
for _, name := range []string{
"users_default", "events_default", "state_default",
"ban", "redact", "kick", "invite",
} {
if raw, ok := obj[name]; ok {
if level, err := parseLevel(raw, rules); err == nil {
out.top[name] = level
}
}
}
for _, name := range []string{"events", "notifications", "users"} {
raw, ok := obj[name]
if !ok {
continue
}
if levels, err := parseLevelMap(raw, rules); err == nil {
out.subs[name] = levels
}
}
return out, nil
}
// unionKeys returns every key in either map, in a stable order so that the
// first rejection reported for a given pair of events is always the same
// one — a rule that rejects differently on different servers is as bad as
// one that rejects differently from the spec.
func unionKeys(a, b map[string]int64) []string {
seen := make(map[string]bool, len(a)+len(b))
keys := make([]string, 0, len(a)+len(b))
for _, m := range []map[string]int64{a, b} {
for k := range m {
if !seen[k] {
seen[k] = true
keys = append(keys, k)
}
}
}
slices.Sort(keys)
return keys
}
// levelString renders a power level for an error message, spelling out the
// one value that is not a number anybody wrote down.
func levelString(level int64) string {
if level == InfinitePowerLevel {
return "infinite (room creator)"
}
return fmt.Sprintf("%d", level)
}
// joinRuleOf reads the join rule from an m.room.join_rules event.
func joinRuleOf(e *event.Event) (string, error) {
var content struct {
JoinRule string `json:"join_rule"`
}
if err := json.Unmarshal(e.Content(), &content); err != nil {
return "", reject("m.room.join_rules content: %v", err)
}
return content.JoinRule, nil
}
// ShouldApplyRedaction reports whether a redaction event takes effect on
// the event it targets.
//
// The authorisation rules let almost any redaction event into the room;
// this is the separate test the spec puts under "Handling redactions",
// which a server can only run once it holds both events. Either the sender
// has the redact level, or the redaction comes from the same server as the
// event it redacts — a server may always take back its own users' events.
//
// Note that the power level test passes even when the sender could not have
// sent the event they are redacting. Moderation is not the same permission
// as authorship.
func ShouldApplyRedaction(redaction, target *event.Event, pl *PowerLevels) bool {
if pl.UserLevel(redaction.Sender()) >= pl.Redact {
return true
}
return id.SameServer(redaction.Sender(), target.Sender())
}
// VerifyThirdPartyInviteSignature reports whether any signature on a
// third-party invite's signed block was made by one of the public keys the
// m.room.third_party_invite event published.
//
// The keys travel in the event itself, so unlike the restricted join rule
// this needs nothing from outside: the identity server's key was recorded
// when the invite was issued.
func VerifyThirdPartyInviteSignature(signed json.RawMessage, inviteEvent *event.Event) bool {
keys, err := thirdPartyInviteKeys(inviteEvent)
if err != nil || len(keys) == 0 {
return false
}
sigs, err := signing.ExtractSignatures(signed)
if err != nil {
return false
}
for serverName, byKeyID := range sigs {
for keyID := range byKeyID {
if algorithm, _, ok := signing.SplitKeyID(keyID); !ok || algorithm != signing.KeyIDPrefix {
continue
}
for _, pub := range keys {
if err := signing.VerifyJSON(serverName, keyID, pub, signed); err == nil {
return true
}
}
}
}
return false
}
File diff suppressed because it is too large. Load diff
+201
View File
@@ -0,0 +1,201 @@
package auth
import (
"encoding/json"
"fmt"
"github.com/dariuszjaniak/warp/server/internal/matrix/event"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
)
// AuthEventKeys returns the state an event must cite in its auth_events,
// per the auth events selection algorithm in the server specification.
//
// The list is what makes an event self-contained: a server receiving it
// years later, with no idea what the room looks like now, can fetch
// exactly these events and re-run the rules. That is also why the set is
// as small as it is — it is the state the rules read, and nothing else.
func AuthEventKeys(e *event.Event) ([]StateKey, error) {
stateKey := (*string)(nil)
if sk, ok := e.StateKey(); ok {
stateKey = &sk
}
return AuthEventKeysFor(e.Rules(), e.Type(), e.Sender(), stateKey, e.Content())
}
// AuthEventKeysFor is AuthEventKeys for an event that does not exist yet,
// so that the roomserver can look up the state it needs before building it.
func AuthEventKeysFor(rules roomversion.Rules, eventType, sender string, stateKey *string, content json.RawMessage) ([]StateKey, error) {
// The create event starts the room; there is nothing for it to cite.
if eventType == event.TypeCreate {
return nil, nil
}
var keys []StateKey
// From room version 12 the create event is left out: the room ID *is*
// its event ID, so every event already refers to it.
if !rules.CreateEventExcludedFromAuthEvents {
keys = append(keys, StateKey{Type: event.TypeCreate})
}
keys = append(keys,
StateKey{Type: event.TypePowerLevels},
StateKey{Type: event.TypeMember, StateKey: sender},
)
if eventType != event.TypeMember {
return keys, nil
}
if stateKey == nil {
return nil, reject("an m.room.member event must have a state_key")
}
keys = append(keys, StateKey{Type: event.TypeMember, StateKey: *stateKey})
var parsed memberContent
if err := json.Unmarshal(content, &parsed); err != nil {
return nil, reject("m.room.member content: %v", err)
}
switch parsed.Membership {
case event.MembershipJoin, event.MembershipInvite, event.MembershipKnock:
keys = append(keys, StateKey{Type: event.TypeJoinRules})
}
if parsed.Membership == event.MembershipInvite && parsed.ThirdPartyInvite != nil {
var signed struct {
Token string `json:"token"`
}
if err := json.Unmarshal(parsed.ThirdPartyInvite.Signed, &signed); err == nil && signed.Token != "" {
keys = append(keys, StateKey{Type: event.TypeThirdPartyInvite, StateKey: signed.Token})
}
}
if parsed.Membership == event.MembershipJoin &&
parsed.JoinAuthorisedViaUsersServer != "" &&
rules.RestrictedJoinRule {
keys = append(keys, StateKey{Type: event.TypeMember, StateKey: parsed.JoinAuthorisedViaUsersServer})
}
return dedupe(keys), nil
}
// CheckAuthEvents applies rule 3 (rule 2 on the version 11 page): the
// auth_events an event declares are well-formed and are the ones the
// selection algorithm asks for.
//
// authEvents are the events the IDs in e.AuthEventIDs() resolve to, with
// any that this server rejected already left out — rule 3.3 asks whether
// an auth event was itself rejected, which only storage can answer, so
// leaving it out of the slice is how the caller reports it. A missing
// entry is then caught here as a citation that does not resolve.
func CheckAuthEvents(e *event.Event, authEvents []*event.Event) error {
if e.Type() == event.TypeCreate {
if len(e.AuthEventIDs()) > 0 {
return reject("an m.room.create event must have no auth_events")
}
return nil
}
allowed, err := AuthEventKeys(e)
if err != nil {
return err
}
allowedSet := make(map[StateKey]bool, len(allowed))
for _, k := range allowed {
allowedSet[k] = true
}
seen := make(map[StateKey]bool, len(authEvents))
hasCreate := false
resolved := 0
for _, auth := range authEvents {
if auth == nil {
// A caller builds this slice from storage lookups, so a nil is
// a citation that resolved to nothing. Counting it as missing
// is both the right answer and cheaper than trusting every
// caller to filter their own misses.
continue
}
resolved++
sk, ok := auth.StateKey()
if !ok {
return reject("auth_events contains %s, which is not a state event", auth.EventID())
}
key := StateKey{Type: auth.Type(), StateKey: sk}
// Rule 3.1: two events for the same piece of state leave it
// ambiguous which one authorised this event.
if seen[key] {
return reject("auth_events has two entries for (%s, %q)", key.Type, key.StateKey)
}
seen[key] = true
// Rule 3.2.
if !allowedSet[key] {
return reject("auth_events contains (%s, %q), which the auth events selection algorithm does not ask for",
key.Type, key.StateKey)
}
// Rule 3.4 (2.5 on the version 11 page): an auth event from
// another room proves nothing about this one.
if auth.RoomID() != e.RoomID() {
return reject("the auth event %s belongs to room %q, not %q", auth.EventID(), auth.RoomID(), e.RoomID())
}
if key.Type == event.TypeCreate {
hasCreate = true
}
}
// Rule 2.4 on the version 11 page; version 12 has no such sub-rule,
// because the create event is not cited at all and its new rule 2 —
// the room ID is the create event's ID — carries the same guarantee.
if !e.Rules().CreateEventExcludedFromAuthEvents && !hasCreate {
return reject("auth_events has no m.room.create event")
}
// Rule 3.3 is the caller's to answer, and this is where their answer
// lands: an ID that resolved to nothing was either never seen or was
// itself rejected, and either way it authorises nothing. The check is
// last so that a specific fault above is reported as itself.
if resolved < len(e.AuthEventIDs()) {
return reject("%d of the %d auth_events are missing or were themselves rejected",
len(e.AuthEventIDs())-resolved, len(e.AuthEventIDs()))
}
return nil
}
// SelectAuthEvents picks the events from a room's state that belong in an
// event's auth_events, ready to be turned into a list of event IDs.
func SelectAuthEvents(rules roomversion.Rules, eventType, sender string, stateKey *string, content json.RawMessage, st State) ([]*event.Event, error) {
keys, err := AuthEventKeysFor(rules, eventType, sender, stateKey, content)
if err != nil {
return nil, err
}
out := make([]*event.Event, 0, len(keys))
for _, k := range keys {
if e := st[k]; e != nil {
out = append(out, e)
}
}
if len(out) > event.MaxAuthEvents {
// The selection algorithm cannot produce this many, so reaching it
// means the algorithm and the limit have drifted apart.
return nil, fmt.Errorf("auth events selection produced %d events, more than the %d an event may carry",
len(out), event.MaxAuthEvents)
}
return out, nil
}
// dedupe removes repeated keys, which happens when a user invites or
// authorises themselves: the sender's member event and the target's are
// then the same piece of state.
func dedupe(keys []StateKey) []StateKey {
seen := make(map[StateKey]bool, len(keys))
out := make([]StateKey, 0, len(keys))
for _, k := range keys {
if seen[k] {
continue
}
seen[k] = true
out = append(out, k)
}
return out
}
+358
View File
@@ -0,0 +1,358 @@
package auth
import (
"context"
"crypto/ed25519"
"encoding/json"
"fmt"
"github.com/dariuszjaniak/warp/server/internal/matrix/event"
"github.com/dariuszjaniak/warp/server/internal/matrix/id"
"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
)
// memberContent is the part of an m.room.member event's content the rules
// read. The pointer and json.RawMessage fields are there so that "absent"
// can be told from "empty", which several of the rules turn on.
type memberContent struct {
Membership string `json:"membership"`
JoinAuthorisedViaUsersServer string `json:"join_authorised_via_users_server"`
ThirdPartyInvite *thirdPartyInvite `json:"third_party_invite"`
}
type thirdPartyInvite struct {
Signed json.RawMessage `json:"signed"`
}
// checkMember applies rule 5, the membership rules.
//
// These are the only rules a user who is not in the room may satisfy, so
// they carry the whole of who can get in: invitations, public rooms,
// knocking, restricted rooms, and who may throw whom out.
func checkMember(ctx context.Context, e *event.Event, st State, create *event.Event, pl *PowerLevels, verifier Verifier) error {
rules := e.Rules()
// Rule 5.1.
target, ok := e.StateKey()
if !ok {
return reject("an m.room.member event must have a state_key")
}
var content memberContent
if err := json.Unmarshal(e.Content(), &content); err != nil {
return reject("m.room.member content: %v", err)
}
if content.Membership == "" {
return reject("an m.room.member event must have a membership")
}
// Rule 5.2: a join authorised by another user must be signed by that
// user's server, not merely by the joining user's. Without it, anybody
// could claim to have been let into a restricted room.
if rules.RestrictedJoinRule && content.JoinAuthorisedViaUsersServer != "" {
if err := checkJoinAuthorisation(ctx, e, content.JoinAuthorisedViaUsersServer, verifier); err != nil {
return err
}
}
senderMembership := st.Membership(e.Sender())
targetMembership := st.Membership(target)
switch content.Membership {
case event.MembershipJoin:
return checkJoin(e, st, create, pl, target, targetMembership, content)
case event.MembershipInvite:
return checkInvite(e, st, pl, target, senderMembership, targetMembership, content)
case event.MembershipLeave:
return checkLeave(e, pl, target, senderMembership, targetMembership)
case event.MembershipBan:
return checkBan(e, pl, target, senderMembership)
case event.MembershipKnock:
if !rules.KnockJoinRule {
return reject("room version %s does not allow knocking", rules.Version)
}
return checkKnock(e, st, target, senderMembership)
default:
// Rule 5.8.
return reject("unknown membership %q", content.Membership)
}
}
// checkJoin applies rule 5.3.
func checkJoin(e *event.Event, st State, create *event.Event, pl *PowerLevels, target, targetMembership string, content memberContent) error {
rules := e.Rules()
// Rule 5.3.1: the creator's own join, the second event in the room.
// There is no state yet to authorise it against, so it is recognised
// by its shape: it follows the create event and nothing else.
prev := e.PrevEventIDs()
if len(prev) == 1 && prev[0] == create.EventID() && target == create.Sender() {
return nil
}
// Rule 5.3.2: you join yourself. Being dragged into a room is an
// invite, which the invited user then accepts with a join of their own.
if e.Sender() != target {
return reject("the sender may not join %q on their behalf", target)
}
// Rule 5.3.3.
if targetMembership == event.MembershipBan {
return reject("the user is banned from the room")
}
joinRule := st.JoinRule()
switch joinRule {
case JoinRuleInvite, JoinRuleKnock:
// Rule 5.3.4. A knock does not admit anybody by itself: it asks,
// and somebody with the power to invite still has to answer.
if targetMembership == event.MembershipInvite || targetMembership == event.MembershipJoin {
return nil
}
return reject("the room's join rule is %q and the user has not been invited (membership is %q)",
joinRule, targetMembership)
case JoinRuleRestricted, JoinRuleKnockRestricted:
if joinRule == JoinRuleRestricted && !rules.RestrictedJoinRule {
return reject("room version %s does not have the restricted join rule", rules.Version)
}
if joinRule == JoinRuleKnockRestricted && !rules.KnockRestrictedJoinRule {
return reject("room version %s does not have the knock_restricted join rule", rules.Version)
}
// Rule 5.3.5.1: an existing invite or join still works, so a
// restricted room is never *harder* to enter than an invite room.
if targetMembership == event.MembershipInvite || targetMembership == event.MembershipJoin {
return nil
}
// Rule 5.3.5.2. The authorising user has to be in the room and
// able to invite, because what they are really doing is vouching
// that the joining user is a member of one of the allowed rooms —
// something this server may be unable to see for itself.
authoriser := content.JoinAuthorisedViaUsersServer
if authoriser == "" {
return reject("the room's join rule is %q and the join is not authorised by any user", joinRule)
}
if m := st.Membership(authoriser); m != event.MembershipJoin {
return reject("the authorising user %q is not joined to the room (membership is %q)", authoriser, m)
}
if level := pl.UserLevel(authoriser); level < pl.Invite {
return reject("the authorising user %q has power level %s, below the invite level %d",
authoriser, levelString(level), pl.Invite)
}
// Rule 5.3.5.3.
return nil
case JoinRulePublic:
// Rule 5.3.6.
return nil
default:
// Rule 5.3.7. "private" lands here, and so does any join rule
// from a future room version: a rule we cannot evaluate must keep
// people out rather than let them in.
return reject("the room's join rule %q does not allow joining", joinRule)
}
}
// checkInvite applies rule 5.4.
func checkInvite(e *event.Event, st State, pl *PowerLevels, target, senderMembership, targetMembership string, content memberContent) error {
// Rule 5.4.1: a third-party invite is an invitation issued to an email
// address or phone number before the server knew which user it was.
// The identity server's signature stands in for the inviter's power.
if content.ThirdPartyInvite != nil {
return checkThirdPartyInvite(e, st, target, targetMembership, content)
}
// Rule 5.4.2.
if senderMembership != event.MembershipJoin {
return reject("the sender is not joined to the room (membership is %q)", senderMembership)
}
// Rule 5.4.3.
if targetMembership == event.MembershipJoin || targetMembership == event.MembershipBan {
return reject("the user is already %q in the room", targetMembership)
}
// Rule 5.4.4.
if level := pl.UserLevel(e.Sender()); level < pl.Invite {
return reject("the sender's power level %s is below the invite level %d", levelString(level), pl.Invite)
}
// Rule 5.4.5 is the negation of the above.
return nil
}
// checkThirdPartyInvite applies rule 5.4.1.
func checkThirdPartyInvite(e *event.Event, st State, target, targetMembership string, content memberContent) error {
// Rule 5.4.1.1.
if targetMembership == event.MembershipBan {
return reject("the user is banned from the room")
}
// Rule 5.4.1.2.
if len(content.ThirdPartyInvite.Signed) == 0 {
return reject("the third_party_invite has no signed property")
}
var signed struct {
MXID string `json:"mxid"`
Token string `json:"token"`
}
if err := json.Unmarshal(content.ThirdPartyInvite.Signed, &signed); err != nil {
return reject("the third_party_invite signed property: %v", err)
}
// Rule 5.4.1.3.
if signed.MXID == "" || signed.Token == "" {
return reject("the third_party_invite signed property needs both mxid and token")
}
// Rule 5.4.1.4.
if signed.MXID != target {
return reject("the third_party_invite names %q but the event invites %q", signed.MXID, target)
}
// Rule 5.4.1.5.
inviteEvent := st.ThirdPartyInvite(signed.Token)
if inviteEvent == nil {
return reject("there is no m.room.third_party_invite in the room state for token %q", signed.Token)
}
// Rule 5.4.1.6: the same user who issued the invitation must be the
// one redeeming it.
if inviteEvent.Sender() != e.Sender() {
return reject("the third-party invite was issued by %q, not by %q", inviteEvent.Sender(), e.Sender())
}
// Rules 5.4.1.7 and 5.4.1.8.
if !VerifyThirdPartyInviteSignature(content.ThirdPartyInvite.Signed, inviteEvent) {
return reject("no signature on the third-party invite matches a public key in %s", inviteEvent.EventID())
}
return nil
}
// checkLeave applies rule 5.5, which covers both leaving and being kicked.
func checkLeave(e *event.Event, pl *PowerLevels, target, senderMembership, targetMembership string) error {
rules := e.Rules()
// Rule 5.5.1: leaving on your own account, which also covers turning
// down an invitation and withdrawing a knock.
if e.Sender() == target {
switch targetMembership {
case event.MembershipInvite, event.MembershipJoin:
return nil
case event.MembershipKnock:
if rules.KnockJoinRule {
return nil
}
}
return reject("the user cannot leave from membership %q", targetMembership)
}
// Rule 5.5.2.
if senderMembership != event.MembershipJoin {
return reject("the sender is not joined to the room (membership is %q)", senderMembership)
}
senderLevel := pl.UserLevel(e.Sender())
// Rule 5.5.3: unbanning is a leave over a ban, and it takes the power
// to have banned in the first place.
if targetMembership == event.MembershipBan && senderLevel < pl.Ban {
return reject("the sender's power level %s is below the ban level %d needed to unban",
levelString(senderLevel), pl.Ban)
}
// Rule 5.5.4.
if senderLevel < pl.Kick {
return reject("the sender's power level %s is below the kick level %d", levelString(senderLevel), pl.Kick)
}
if targetLevel := pl.UserLevel(target); targetLevel >= senderLevel {
return reject("the sender's power level %s does not exceed %q's power level %s",
levelString(senderLevel), target, levelString(targetLevel))
}
// Rule 5.5.5 is the negation of the above.
return nil
}
// checkBan applies rule 5.6.
func checkBan(e *event.Event, pl *PowerLevels, target, senderMembership string) error {
// Rule 5.6.1.
if senderMembership != event.MembershipJoin {
return reject("the sender is not joined to the room (membership is %q)", senderMembership)
}
// Rule 5.6.2.
senderLevel := pl.UserLevel(e.Sender())
if senderLevel < pl.Ban {
return reject("the sender's power level %s is below the ban level %d", levelString(senderLevel), pl.Ban)
}
if targetLevel := pl.UserLevel(target); targetLevel >= senderLevel {
return reject("the sender's power level %s does not exceed %q's power level %s",
levelString(senderLevel), target, levelString(targetLevel))
}
// Rule 5.6.3 is the negation of the above.
return nil
}
// checkKnock applies rule 5.7.
func checkKnock(e *event.Event, st State, target, senderMembership string) error {
// Rule 5.7.1.
joinRule := st.JoinRule()
if joinRule != JoinRuleKnock && joinRule != JoinRuleKnockRestricted {
return reject("the room's join rule is %q, so it cannot be knocked on", joinRule)
}
// Rule 5.7.2.
if e.Sender() != target {
return reject("the sender may not knock on behalf of %q", target)
}
// Rule 5.7.3: knocking is allowed from anything but banned, invited or
// joined — there is no point knocking on a door that is already open.
switch senderMembership {
case event.MembershipBan, event.MembershipInvite, event.MembershipJoin:
// Rule 5.7.4.
return reject("the user cannot knock while their membership is %q", senderMembership)
default:
return nil
}
}
// checkJoinAuthorisation applies rule 5.2.
func checkJoinAuthorisation(ctx context.Context, e *event.Event, authoriser string, verifier Verifier) error {
serverName, err := id.ServerNameOf(authoriser)
if err != nil {
return reject("join_authorised_via_users_server: %v", err)
}
if verifier == nil {
return fmt.Errorf("%w: %s is authorised by %q, whose server's signature cannot be checked",
ErrNoVerifier, e.EventID(), authoriser)
}
if err := verifier.VerifyEventSignature(ctx, serverName, e); err != nil {
return reject("the join is authorised by %q but carries no valid signature from %q: %v",
authoriser, serverName, err)
}
return nil
}
// thirdPartyInviteKeys returns the public keys an m.room.third_party_invite
// event published, from either of the two places the spec allows.
func thirdPartyInviteKeys(e *event.Event) ([]ed25519.PublicKey, error) {
var content struct {
PublicKey string `json:"public_key"`
PublicKeys []struct {
PublicKey string `json:"public_key"`
} `json:"public_keys"`
}
if err := json.Unmarshal(e.Content(), &content); err != nil {
return nil, reject("m.room.third_party_invite content: %v", err)
}
encoded := make([]string, 0, len(content.PublicKeys)+1)
if content.PublicKey != "" {
encoded = append(encoded, content.PublicKey)
}
for _, k := range content.PublicKeys {
if k.PublicKey != "" {
encoded = append(encoded, k.PublicKey)
}
}
keys := make([]ed25519.PublicKey, 0, len(encoded))
for _, s := range encoded {
raw, err := signing.DecodeBase64(s)
if err != nil || len(raw) != ed25519.PublicKeySize {
// A key we cannot read cannot match a signature, and the rule
// asks whether *any* key matches — so a bad one is skipped
// rather than fatal.
continue
}
keys = append(keys, ed25519.PublicKey(raw))
}
return keys, nil
}
+270
View File
@@ -0,0 +1,270 @@
package auth
import (
"encoding/json"
"fmt"
"math"
"strconv"
"github.com/dariuszjaniak/warp/server/internal/matrix/event"
"github.com/dariuszjaniak/warp/server/internal/matrix/id"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
)
// InfinitePowerLevel is the level a room creator has from room version 12.
//
// The spec calls it "infinitely high". Since every level that can be
// written down is capped at 2^53-1 by canonical JSON, the largest int64
// stands in for infinity without any comparison going wrong — and nothing
// here does arithmetic on a level, only compares them.
const InfinitePowerLevel int64 = math.MaxInt64
// The default levels, which apply key by key: a power levels event that
// sets only "users" leaves every other level at the value below.
//
// state_default is 50 even in a room with no m.room.power_levels event at
// all, which looks odd until you notice that the creator has 100 there, so
// the only person who can send state is the one person who should be able
// to.
const (
DefaultBan int64 = 50
DefaultKick int64 = 50
DefaultRedact int64 = 50
DefaultInvite int64 = 0
DefaultEventsDefault int64 = 0
DefaultStateDefault int64 = 50
DefaultUsersDefault int64 = 0
DefaultNotification int64 = 50
// CreatorPowerLevel is what the room's creator has while the room has
// no m.room.power_levels event, in room versions up to 11.
CreatorPowerLevel int64 = 100
)
// PowerLevels is a room's resolved power level configuration: the event's
// content with every default filled in, and the room's creators noted so
// that their level can be answered.
type PowerLevels struct {
Ban int64
Kick int64
Redact int64
Invite int64
EventsDefault int64
StateDefault int64
UsersDefault int64
Events map[string]int64
Notifications map[string]int64
Users map[string]int64
// creators have InfinitePowerLevel from room version 12. Before that
// the map is empty and the creator's 100 is written into Users
// instead, because it is a real level there that a later power levels
// event can change.
creators map[string]bool
}
// ResolvePowerLevels works out the levels in force, given the room's
// m.room.power_levels event (which may be nil) and its create event.
//
// The create event is needed even when there is a power levels event: from
// room version 12 it is what says who the creators are, and their level
// cannot be written down.
func ResolvePowerLevels(plEvent, createEvent *event.Event, rules roomversion.Rules) (*PowerLevels, error) {
pl := &PowerLevels{
Ban: DefaultBan,
Kick: DefaultKick,
Redact: DefaultRedact,
Invite: DefaultInvite,
EventsDefault: DefaultEventsDefault,
StateDefault: DefaultStateDefault,
UsersDefault: DefaultUsersDefault,
Events: map[string]int64{},
Notifications: map[string]int64{},
Users: map[string]int64{},
creators: map[string]bool{},
}
if createEvent != nil {
creators, err := Creators(createEvent)
if err != nil {
return nil, err
}
if rules.CreatorsHaveInfinitePower {
for _, c := range creators {
pl.creators[c] = true
}
} else if plEvent == nil && len(creators) > 0 {
// With no power levels event, the creator has 100 and everyone
// else has nothing.
pl.Users[creators[0]] = CreatorPowerLevel
}
}
if plEvent == nil {
return pl, nil
}
if err := pl.apply(plEvent.Content(), rules); err != nil {
return nil, err
}
return pl, nil
}
// apply overlays the content of an m.room.power_levels event.
func (p *PowerLevels) apply(content json.RawMessage, rules roomversion.Rules) error {
var obj map[string]json.RawMessage
if err := json.Unmarshal(content, &obj); err != nil {
return reject("m.room.power_levels content is not an object: %v", err)
}
scalars := map[string]*int64{
"ban": &p.Ban,
"kick": &p.Kick,
"redact": &p.Redact,
"invite": &p.Invite,
"events_default": &p.EventsDefault,
"state_default": &p.StateDefault,
"users_default": &p.UsersDefault,
}
for name, target := range scalars {
raw, ok := obj[name]
if !ok {
continue
}
level, err := parseLevel(raw, rules)
if err != nil {
return reject("m.room.power_levels %s: %v", name, err)
}
*target = level
}
for name, target := range map[string]*map[string]int64{
"events": &p.Events,
"notifications": &p.Notifications,
"users": &p.Users,
} {
raw, ok := obj[name]
if !ok {
continue
}
levels, err := parseLevelMap(raw, rules)
if err != nil {
return reject("m.room.power_levels %s: %v", name, err)
}
if name == "users" {
for userID := range levels {
if err := id.ValidateUserID(userID); err != nil {
return reject("m.room.power_levels users: %v", err)
}
}
}
*target = levels
}
return nil
}
// UserLevel returns a user's power level.
func (p *PowerLevels) UserLevel(userID string) int64 {
if p.creators[userID] {
return InfinitePowerLevel
}
if level, ok := p.Users[userID]; ok {
return level
}
return p.UsersDefault
}
// IsCreator reports whether a user is one of the room's creators, in a room
// version where that gives them infinite power.
func (p *PowerLevels) IsCreator(userID string) bool { return p.creators[userID] }
// SendLevel returns the level needed to send an event of a type. The events
// map overrides everything; otherwise a state event needs state_default and
// a message event events_default.
func (p *PowerLevels) SendLevel(eventType string, isState bool) int64 {
if level, ok := p.Events[eventType]; ok {
return level
}
if isState {
return p.StateDefault
}
return p.EventsDefault
}
// NotificationLevel returns the level needed to trigger a notification,
// such as "room" for an @room mention.
func (p *PowerLevels) NotificationLevel(key string) int64 {
if level, ok := p.Notifications[key]; ok {
return level
}
return DefaultNotification
}
// Creators returns the user IDs that count as creators of the room: the
// sender of the create event, plus content.additional_creators in room
// versions that allow it.
func Creators(createEvent *event.Event) ([]string, error) {
if createEvent.Type() != event.TypeCreate {
return nil, fmt.Errorf("%s is not an m.room.create event", createEvent.EventID())
}
creators := []string{createEvent.Sender()}
if !createEvent.Rules().AdditionalCreators {
return creators, nil
}
var content struct {
AdditionalCreators []string `json:"additional_creators"`
}
if err := json.Unmarshal(createEvent.Content(), &content); err != nil {
return nil, reject("m.room.create additional_creators is not an array of strings: %v", err)
}
for _, c := range content.AdditionalCreators {
if err := id.ValidateUserID(c); err != nil {
return nil, reject("m.room.create additional_creators: %v", err)
}
creators = append(creators, c)
}
return creators, nil
}
// parseLevel reads a single power level.
//
// Room versions up to 9 accepted a level written as a string, so "50" and
// 50 both meant fifty — a laxness that let two servers that disagreed
// about how to parse a string disagree about who could speak. Version 10
// requires an integer.
func parseLevel(raw json.RawMessage, rules roomversion.Rules) (int64, error) {
var level int64
if err := json.Unmarshal(raw, &level); err == nil {
return level, nil
}
if rules.RequireIntegerPowerLevels {
return 0, fmt.Errorf("%s is not an integer", raw)
}
var s string
if err := json.Unmarshal(raw, &s); err != nil {
return 0, fmt.Errorf("%s is neither an integer nor a string", raw)
}
level, err := strconv.ParseInt(s, 10, 64)
if err != nil {
return 0, fmt.Errorf("%s is a string that is not an integer", raw)
}
return level, nil
}
// parseLevelMap reads the events, notifications or users map.
func parseLevelMap(raw json.RawMessage, rules roomversion.Rules) (map[string]int64, error) {
var obj map[string]json.RawMessage
if err := json.Unmarshal(raw, &obj); err != nil {
return nil, fmt.Errorf("not an object: %v", err)
}
out := make(map[string]int64, len(obj))
for k, v := range obj {
level, err := parseLevel(v, rules)
if err != nil {
return nil, fmt.Errorf("%q: %w", k, err)
}
out[k] = level
}
return out, nil
}
+95
View File
@@ -0,0 +1,95 @@
package auth
import (
"github.com/dariuszjaniak/warp/server/internal/matrix/event"
)
// StateKey identifies one piece of room state: an event type and a state
// key together. Room state is a map from this to an event.
type StateKey struct {
Type string
StateKey string
}
// State is the room state an event is authorised against.
//
// It is only ever the handful of events the rules actually read — the
// create event, the power levels, the join rules, the members involved and
// any third-party invite — not the whole state of the room. Two callers
// build it differently: event intake builds it from the event's own
// auth_events, and state resolution builds it from the state it has
// resolved so far.
type State map[StateKey]*event.Event
// NewState indexes a set of state events.
//
// A non-state event in the input is ignored rather than refused: the usual
// source is an event's auth_events, and a server that cited something odd
// there should fail the auth_events checks, not crash the indexer.
func NewState(events []*event.Event) State {
st := make(State, len(events))
for _, e := range events {
if sk, ok := e.StateKey(); ok {
st[StateKey{Type: e.Type(), StateKey: sk}] = e
}
}
return st
}
// Get returns one state event, or nil.
func (s State) Get(eventType, stateKey string) *event.Event {
return s[StateKey{Type: eventType, StateKey: stateKey}]
}
// Create returns the m.room.create event, or nil if the state does not
// contain one — which for any event but the create event itself means the
// event cannot be authorised at all.
func (s State) Create() *event.Event { return s.Get(event.TypeCreate, "") }
// PowerLevels returns the m.room.power_levels event, or nil. A room
// without one is not broken: until the creator sends one, the defaults
// apply and the creator alone has power.
func (s State) PowerLevels() *event.Event { return s.Get(event.TypePowerLevels, "") }
// JoinRules returns the m.room.join_rules event, or nil.
func (s State) JoinRules() *event.Event { return s.Get(event.TypeJoinRules, "") }
// Member returns a user's m.room.member event, or nil.
func (s State) Member(userID string) *event.Event { return s.Get(event.TypeMember, userID) }
// ThirdPartyInvite returns the m.room.third_party_invite event for a token,
// or nil.
func (s State) ThirdPartyInvite(token string) *event.Event {
return s.Get(event.TypeThirdPartyInvite, token)
}
// Membership returns a user's membership in the room.
//
// A user with no m.room.member event has never been in the room, which the
// rules treat exactly like having left: every rule that cares asks whether
// the membership *is* something, never whether it is absent.
func (s State) Membership(userID string) string {
e := s.Member(userID)
if e == nil {
return event.MembershipLeave
}
membership, err := e.Membership()
if err != nil || membership == "" {
return event.MembershipLeave
}
return membership
}
// JoinRule returns the room's join rule, defaulting to "invite" as the
// spec does for a room with no m.room.join_rules event.
func (s State) JoinRule() string {
e := s.JoinRules()
if e == nil {
return JoinRuleInvite
}
rule, err := joinRuleOf(e)
if err != nil || rule == "" {
return JoinRuleInvite
}
return rule
}
@@ -0,0 +1,263 @@
// Package canonicaljson implements the canonical JSON encoding Matrix uses
// wherever two servers have to agree on bytes rather than on values: event
// content hashes, event IDs, and every signature.
//
// The rules (spec appendices, "Canonical JSON") are short. Sort object keys
// by Unicode code point, emit no whitespace, allow only integers in
// [-(2^53)+1, 2^53-1], and escape as little as possible — non-ASCII text
// goes out as raw UTF-8 rather than \u escapes.
//
// encoding/json cannot produce this on its own: it escapes <, > and & so
// that output is safe to paste into HTML, which no other Matrix
// implementation does, and it reads and writes floats happily. So the
// encoder below is hand-written and encoding/json is used only to parse.
package canonicaljson
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"slices"
"strconv"
"strings"
"unicode/utf8"
)
// The range Matrix allows for integers, which is the range a float64 holds
// without losing precision — the limit comes from JavaScript, like so much
// else about JSON.
const (
MaxInt int64 = 1<<53 - 1
MinInt int64 = -(1<<53 - 1)
)
// ErrNotCanonical is the root of every error this package returns, so that
// a caller who only wants to answer M_BAD_JSON can test for it.
var ErrNotCanonical = errors.New("not canonical JSON")
// Encode canonicalises a JSON value.
//
// Input need not already be canonical — keys may be in any order and
// strings may use any legal escaping — but it must be *encodable* as
// canonical JSON, so a float or an oversized integer is an error rather
// than something to round off.
func Encode(raw []byte) ([]byte, error) {
v, err := parse(raw)
if err != nil {
return nil, err
}
buf := bytes.NewBuffer(make([]byte, 0, len(raw)))
if err := write(buf, v); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// Marshal encodes v as JSON and canonicalises the result.
func Marshal(v any) ([]byte, error) {
raw, err := json.Marshal(v)
if err != nil {
return nil, fmt.Errorf("marshal JSON: %w", err)
}
return Encode(raw)
}
// Valid reports whether raw can be encoded as canonical JSON, with an error
// that says what is wrong with it if it cannot.
func Valid(raw []byte) error {
_, err := Encode(raw)
return err
}
// parse reads raw into the generic Go representation, keeping numbers as
// their source text so that write can decide what to do with them.
func parse(raw []byte) (any, error) {
if !utf8.Valid(raw) {
return nil, fmt.Errorf("%w: not valid UTF-8", ErrNotCanonical)
}
if err := checkEscapes(raw); err != nil {
return nil, err
}
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var v any
if err := dec.Decode(&v); err != nil {
return nil, fmt.Errorf("%w: %w", ErrNotCanonical, err)
}
// A signature covers one value. Anything after it is either a second
// document or junk, and either way the bytes we would hash are not the
// bytes we were handed.
if dec.More() {
return nil, fmt.Errorf("%w: trailing data after the JSON value", ErrNotCanonical)
}
return v, nil
}
func write(buf *bytes.Buffer, v any) error {
switch v := v.(type) {
case nil:
buf.WriteString("null")
case bool:
if v {
buf.WriteString("true")
} else {
buf.WriteString("false")
}
case json.Number:
return writeNumber(buf, v)
case string:
writeString(buf, v)
case []any:
buf.WriteByte('[')
for i, elem := range v {
if i > 0 {
buf.WriteByte(',')
}
if err := write(buf, elem); err != nil {
return err
}
}
buf.WriteByte(']')
case map[string]any:
keys := make([]string, 0, len(v))
for k := range v {
keys = append(keys, k)
}
// Go compares strings byte by byte, and UTF-8 is ordered so that a
// byte-wise comparison is a code point comparison. Sorting the keys
// as Go strings is therefore exactly what the spec asks for.
slices.Sort(keys)
buf.WriteByte('{')
for i, k := range keys {
if i > 0 {
buf.WriteByte(',')
}
writeString(buf, k)
buf.WriteByte(':')
if err := write(buf, v[k]); err != nil {
return err
}
}
buf.WriteByte('}')
default:
return fmt.Errorf("%w: cannot encode a value of type %T", ErrNotCanonical, v)
}
return nil
}
// writeNumber re-emits a number from its source text.
//
// Matrix has no floats, so anything with a decimal point or an exponent is
// rejected rather than rounded: a server that quietly turned 1.0 into 1
// would compute a hash nobody else agrees with. The value is reformatted
// from the parsed integer, which also normalises the one spelling the
// grammar forbids but parsers accept, "-0".
func writeNumber(buf *bytes.Buffer, n json.Number) error {
s := string(n)
if strings.ContainsAny(s, ".eE") {
return fmt.Errorf("%w: %s is not an integer; Matrix JSON has no floats", ErrNotCanonical, s)
}
i, err := strconv.ParseInt(s, 10, 64)
if err != nil {
return fmt.Errorf("%w: %s is not an integer we can represent", ErrNotCanonical, s)
}
if i > MaxInt || i < MinInt {
return fmt.Errorf("%w: %s is outside the allowed range [%d, %d]", ErrNotCanonical, s, MinInt, MaxInt)
}
buf.WriteString(strconv.FormatInt(i, 10))
return nil
}
const hexDigits = "0123456789abcdef"
// writeString writes a JSON string literal with the minimum of escaping:
// the two characters that must always be escaped, the five control
// characters that have a short form, and \u00XX for the remaining controls.
// Everything else, non-ASCII included, is copied through as UTF-8.
func writeString(buf *bytes.Buffer, s string) {
buf.WriteByte('"')
for i := 0; i < len(s); i++ {
switch c := s[i]; c {
case '"':
buf.WriteString(`\"`)
case '\\':
buf.WriteString(`\\`)
case '\b':
buf.WriteString(`\b`)
case '\f':
buf.WriteString(`\f`)
case '\n':
buf.WriteString(`\n`)
case '\r':
buf.WriteString(`\r`)
case '\t':
buf.WriteString(`\t`)
default:
if c < 0x20 {
buf.WriteString(`\u00`)
buf.WriteByte(hexDigits[c>>4])
buf.WriteByte(hexDigits[c&0xf])
continue
}
buf.WriteByte(c)
}
}
buf.WriteByte('"')
}
// checkEscapes rejects \u escapes that do not form a character.
//
// encoding/json turns a lone surrogate into U+FFFD instead of failing,
// which would quietly change the bytes we are about to hash — Python's
// json, which the spec uses as its reference encoder, raises instead.
// Scanning the source for the escape is the only way to tell the two cases
// apart once the parser has run.
//
// Walking from the first byte is safe because a backslash can only appear
// inside a string, so every one found here opens an escape sequence.
func checkEscapes(raw []byte) error {
for i := 0; i < len(raw); i++ {
if raw[i] != '\\' {
continue
}
if i+1 >= len(raw) {
break // a truncated escape; the parser reports it better than we can
}
if raw[i+1] != 'u' {
i++ // an escape of one character, including a backslash itself
continue
}
r, ok := hexEscape(raw, i)
if !ok {
break
}
switch {
case r >= 0xD800 && r <= 0xDBFF:
low, ok := hexEscape(raw, i+6)
if !ok || low < 0xDC00 || low > 0xDFFF {
return fmt.Errorf("%w: \\u%04X is a high surrogate with no low surrogate after it", ErrNotCanonical, r)
}
i += 11
case r >= 0xDC00 && r <= 0xDFFF:
return fmt.Errorf("%w: \\u%04X is a low surrogate with no high surrogate before it", ErrNotCanonical, r)
default:
i += 5
}
}
return nil
}
// hexEscape reads the \uXXXX escape starting at i, if that is what is there.
func hexEscape(raw []byte, i int) (rune, bool) {
if i+6 > len(raw) || raw[i] != '\\' || raw[i+1] != 'u' {
return 0, false
}
v, err := strconv.ParseUint(string(raw[i+2:i+6]), 16, 32)
if err != nil {
return 0, false
}
return rune(v), true
}
@@ -0,0 +1,292 @@
package canonicaljson
import (
"errors"
"strings"
"testing"
)
// TestSpecExamples runs the worked examples from the spec appendices,
// section "Canonical JSON". They are the one set of values every
// implementation is expected to agree on.
func TestSpecExamples(t *testing.T) {
cases := []struct {
name string
in string
want string
}{
{"empty object", `{}`, `{}`},
{"values", "{\n \"one\": 1,\n \"two\": \"Two\"\n}", `{"one":1,"two":"Two"}`},
{"sorts keys", "{\n \"b\": \"2\",\n \"a\": \"1\"\n}", `{"a":"1","b":"2"}`},
{"sorts keys, no whitespace to remove", `{"b":"2","a":"1"}`, `{"a":"1","b":"2"}`},
{
"nested",
`{
"auth": {
"success": true,
"mxid": "@john.doe:example.com",
"profile": {
"display_name": "John Doe",
"three_pids": [
{"medium": "email", "address": "john.doe@example.org"},
{"medium": "msisdn", "address": "123456789"}
]
}
}
}`,
`{"auth":{"mxid":"@john.doe:example.com","profile":{"display_name":"John Doe",` +
`"three_pids":[{"address":"john.doe@example.org","medium":"email"},` +
`{"address":"123456789","medium":"msisdn"}]},"success":true}}`,
},
{"non-ASCII stays UTF-8", `{"a": "日本語"}`, `{"a":"日本語"}`},
{"non-ASCII keys sort by code point", `{"本":2,"日":1}`, `{"日":1,"本":2}`},
{"escapes are decoded", `{"a": "日"}`, `{"a":"日"}`},
{"null", `{"a": null}`, `{"a":null}`},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, err := Encode([]byte(tc.in))
if err != nil {
t.Fatalf("Encode: %v", err)
}
if string(got) != tc.want {
t.Errorf("Encode() = %s, want %s", got, tc.want)
}
})
}
}
// TestEncodeIsIdempotent checks that canonicalising canonical JSON changes
// nothing, which is what lets a stored event be re-signed and re-hashed
// without drifting.
func TestEncodeIsIdempotent(t *testing.T) {
once, err := Encode([]byte(`{"b":[1,2,{"d":"日","c":null}],"a":true}`))
if err != nil {
t.Fatal(err)
}
twice, err := Encode(once)
if err != nil {
t.Fatal(err)
}
if string(once) != string(twice) {
t.Errorf("second pass changed the output:\n %s\n %s", once, twice)
}
}
// TestNoHTMLEscaping is the reason this package exists rather than calling
// encoding/json: the standard library escapes these three characters, and
// an event containing one would hash differently on every other server.
func TestNoHTMLEscaping(t *testing.T) {
got, err := Encode([]byte(`{"body":"<b> & </b>"}`))
if err != nil {
t.Fatal(err)
}
if want := `{"body":"<b> & </b>"}`; string(got) != want {
t.Errorf("Encode() = %s, want %s", got, want)
}
}
// TestStringEscaping checks the escapes the grammar allows: the short forms
// where they exist and \u00XX for the other control characters, with
// everything else passed through.
func TestStringEscaping(t *testing.T) {
cases := []struct {
name string
in string
want string
}{
{"quote and backslash", `{"a":"\" \\"}`, `{"a":"\" \\"}`},
{"short forms", `{"a":"\b\t\n\f\r"}`, `{"a":"\b\t\n\f\r"}`},
{"other controls", `{"a":"\u0000\u001f\u000b\u000e"}`, `{"a":"\u0000\u001f\u000b\u000e"}`},
{"literal newline is not legal JSON input", "{\"a\":\"\t\"}", ""},
{"solidus is not escaped", `{"a":"/"}`, `{"a":"/"}`},
{"surrogate pair", `{"a":"😀"}`, `{"a":"😀"}`},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, err := Encode([]byte(tc.in))
if tc.want == "" {
if err == nil {
t.Fatalf("Encode(%s) = %s, want an error", tc.in, got)
}
return
}
if err != nil {
t.Fatalf("Encode: %v", err)
}
if string(got) != tc.want {
t.Errorf("Encode() = %s, want %s", got, tc.want)
}
})
}
}
func TestNumbers(t *testing.T) {
cases := []struct {
name string
in string
want string
wantErr string
}{
{name: "zero", in: `{"a":0}`, want: `{"a":0}`},
{name: "negative", in: `{"a":-12}`, want: `{"a":-12}`},
{name: "largest allowed", in: `{"a":9007199254740991}`, want: `{"a":9007199254740991}`},
{name: "smallest allowed", in: `{"a":-9007199254740991}`, want: `{"a":-9007199254740991}`},
// The grammar forbids -0 in the output, so it is normalised rather
// than rejected: every JSON parser reads it as zero anyway.
{name: "negative zero is normalised", in: `{"a":-0}`, want: `{"a":0}`},
{name: "float", in: `{"a":1.0}`, wantErr: "has no floats"},
{name: "fraction", in: `{"a":0.5}`, wantErr: "has no floats"},
{name: "exponent", in: `{"a":1e3}`, wantErr: "has no floats"},
{name: "above the range", in: `{"a":9007199254740992}`, wantErr: "outside the allowed range"},
{name: "below the range", in: `{"a":-9007199254740992}`, wantErr: "outside the allowed range"},
{name: "far above the range", in: `{"a":18446744073709551616}`, wantErr: "not an integer we can represent"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, err := Encode([]byte(tc.in))
if tc.wantErr != "" {
if err == nil {
t.Fatalf("Encode(%s) = %s, want an error", tc.in, got)
}
if !strings.Contains(err.Error(), tc.wantErr) {
t.Errorf("error = %q, want it to mention %q", err, tc.wantErr)
}
if !errors.Is(err, ErrNotCanonical) {
t.Errorf("error does not wrap ErrNotCanonical: %v", err)
}
return
}
if err != nil {
t.Fatalf("Encode: %v", err)
}
if string(got) != tc.want {
t.Errorf("Encode() = %s, want %s", got, tc.want)
}
})
}
}
func TestRejects(t *testing.T) {
cases := []struct {
name string
in string
want string
}{
{"invalid UTF-8", "{\"a\":\"\xff\xfe\"}", "not valid UTF-8"},
{"lone high surrogate", `{"a":"\uD800"}`, "high surrogate"},
{"lone low surrogate", `{"a":"\uDC00"}`, "low surrogate"},
{"high surrogate then a plain character", `{"a":"\uD800x"}`, "high surrogate"},
{"trailing data", `{"a":1} {"b":2}`, "trailing data"},
{"not JSON at all", `nope`, "not canonical JSON"},
{"empty input", ``, "not canonical JSON"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, err := Encode([]byte(tc.in))
if err == nil {
t.Fatalf("Encode(%q) = %s, want an error", tc.in, got)
}
if !strings.Contains(err.Error(), tc.want) {
t.Errorf("error = %q, want it to mention %q", err, tc.want)
}
if !errors.Is(err, ErrNotCanonical) {
t.Errorf("error does not wrap ErrNotCanonical: %v", err)
}
})
}
}
// TestEscapedBackslashIsNotAnEscape guards the backslash counting in
// checkEscapes: the sequence below is a literal backslash followed by the
// letters uD800, not a lone surrogate.
func TestEscapedBackslashIsNotAnEscape(t *testing.T) {
got, err := Encode([]byte(`{"a":"\\uD800"}`))
if err != nil {
t.Fatalf("Encode: %v", err)
}
if want := `{"a":"\\uD800"}`; string(got) != want {
t.Errorf("Encode() = %s, want %s", got, want)
}
}
// TestNonObjectValues checks that the encoder handles every JSON type at
// the top level, not just objects. Signing only ever sees objects, but
// hashing a bare array should not be a panic.
func TestNonObjectValues(t *testing.T) {
for _, tc := range []struct{ in, want string }{
{`[1, 2, 3]`, `[1,2,3]`},
{`[]`, `[]`},
{` "x" `, `"x"`},
{`true`, `true`},
{`null`, `null`},
{`7`, `7`},
} {
got, err := Encode([]byte(tc.in))
if err != nil {
t.Fatalf("Encode(%s): %v", tc.in, err)
}
if string(got) != tc.want {
t.Errorf("Encode(%s) = %s, want %s", tc.in, got, tc.want)
}
}
}
func TestMarshal(t *testing.T) {
// A Go value with its fields in the wrong order, and a string the
// standard library would HTML-escape.
v := struct {
Two string `json:"two"`
One int `json:"one"`
Body string `json:"body"`
}{Two: "Two", One: 1, Body: "a & b"}
got, err := Marshal(v)
if err != nil {
t.Fatalf("Marshal: %v", err)
}
if want := `{"body":"a & b","one":1,"two":"Two"}`; string(got) != want {
t.Errorf("Marshal() = %s, want %s", got, want)
}
}
func TestValid(t *testing.T) {
if err := Valid([]byte(`{"a":1}`)); err != nil {
t.Errorf("Valid: %v", err)
}
if err := Valid([]byte(`{"a":1.5}`)); err == nil {
t.Error("Valid accepted a float")
}
}
// FuzzEncode checks the two properties that matter for anything arriving
// over the network: the encoder never panics, and whatever it accepts it
// accepts again unchanged. The second half is what guarantees two servers
// agree on the bytes they hash.
func FuzzEncode(f *testing.F) {
for _, seed := range []string{
`{}`, `{"a":1}`, `{"a":"日本語"}`, `[1,2,3]`, `{"a":{"b":[null,true]}}`,
`{"a":"\u0000"}`, `{"a":-0}`, `{"a":"😀"}`, `0`, `""`,
} {
f.Add([]byte(seed))
}
f.Fuzz(func(t *testing.T, in []byte) {
once, err := Encode(in)
if err != nil {
return
}
twice, err := Encode(once)
if err != nil {
t.Fatalf("re-encoding canonical JSON failed: %v\ninput: %q\ncanonical: %s", err, in, once)
}
if string(once) != string(twice) {
t.Fatalf("not idempotent:\ninput: %q\nonce: %s\ntwice: %s", in, once, twice)
}
})
}
+203
View File
@@ -0,0 +1,203 @@
package event
import (
"encoding/json"
"fmt"
"time"
"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
"github.com/dariuszjaniak/warp/server/internal/matrix/id"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
)
// Builder assembles a new event for this server to send.
//
// The caller fills in what the event says; Build fills in what the room
// version says — where the redaction target goes, whether the create event
// carries a room ID — then hashes, signs and validates the result. The
// roomserver (PLAN.md phase 3) is the only thing that should build events,
// because it is the only thing that knows the right prev_events, depth and
// auth_events.
type Builder struct {
// RoomID is the room the event belongs to. It must be empty for an
// m.room.create event in a room version where the room ID is derived
// from that event; use RoomIDOfCreateEvent on the built event to find
// out what the room ID turned out to be.
RoomID string
// Type is the event type, such as "m.room.message".
Type string
// StateKey makes this a state event when it is non-nil. The empty
// string is a perfectly good state key, which is why this is a pointer.
StateKey *string
// Sender is the user ID sending the event. Its server name is the one
// the event is signed as, since an event must be signed by its
// sender's server.
Sender string
// Content is the event content. An empty Content becomes {}.
Content json.RawMessage
// PrevEvents and AuthEvents are the event IDs this event refers to.
PrevEvents []string
AuthEvents []string
// Depth is one more than the greatest depth of the prev_events.
Depth int64
// Redacts is the event an m.room.redaction event targets. Build puts
// it wherever this room version keeps it, so callers do not have to
// know which versions moved it into the content.
Redacts string
// Unsigned is attached to the event but covered by neither its hashes
// nor its signatures, so anything here can be changed in transit by
// any server that passes the event on.
Unsigned json.RawMessage
}
// Build hashes, signs and validates the event.
//
// now becomes origin_server_ts. It is passed in rather than read from the
// clock so that tests are deterministic and so that a caller sending a
// batch can stamp them all alike.
func (b *Builder) Build(now time.Time, key *signing.KeyPair, rules roomversion.Rules) (*Event, error) {
serverName, err := id.ServerNameOf(b.Sender)
if err != nil {
return nil, fmt.Errorf("build event: sender: %w", err)
}
if b.Type == "" {
return nil, fmt.Errorf("build event: type is required")
}
if b.Depth < 0 {
return nil, fmt.Errorf("build event: depth %d is negative", b.Depth)
}
obj := map[string]json.RawMessage{}
set := func(k string, v any) error {
raw, err := json.Marshal(v)
if err != nil {
return fmt.Errorf("build event: %s: %w", k, err)
}
obj[k] = raw
return nil
}
content := b.Content
if len(content) == 0 {
content = json.RawMessage("{}")
}
if b.Redacts != "" {
if content, err = b.placeRedacts(content, obj, rules); err != nil {
return nil, err
}
}
if err := b.setRoomID(obj, rules); err != nil {
return nil, err
}
for k, v := range map[string]any{
"type": b.Type,
"sender": b.Sender,
"depth": b.Depth,
"origin_server_ts": now.UnixMilli(),
// prev_events and auth_events are required, and must be [] rather
// than absent when there are none — which is the case for the
// create event and for nothing else.
"prev_events": orEmpty(b.PrevEvents),
"auth_events": orEmpty(b.AuthEvents),
} {
if err := set(k, v); err != nil {
return nil, err
}
}
obj["content"] = content
if b.StateKey != nil {
if err := set("state_key", *b.StateKey); err != nil {
return nil, err
}
}
if len(b.Unsigned) > 0 {
obj["unsigned"] = b.Unsigned
}
raw, err := canonicaljson.Marshal(obj)
if err != nil {
return nil, fmt.Errorf("build event: %w", err)
}
signed, err := hashAndSign(raw, rules, serverName, key)
if err != nil {
return nil, fmt.Errorf("build event: %w", err)
}
// Parse is the one gate every event goes through, built here or
// received over federation, so a bug in the builder surfaces as a
// failure to build rather than as an invalid event in a room.
return Parse(signed, rules)
}
// setRoomID writes the room_id property, or leaves it out for a create
// event in a room version that derives the room ID from it.
func (b *Builder) setRoomID(obj map[string]json.RawMessage, rules roomversion.Rules) error {
if rules.RoomIDIsCreateEventID && b.Type == TypeCreate {
if b.RoomID != "" {
return fmt.Errorf("build event: an m.room.create event in a version %s room must not have a room_id; it becomes the room ID",
rules.Version)
}
return nil
}
if err := id.ValidateRoomID(b.RoomID); err != nil {
return fmt.Errorf("build event: room_id: %w", err)
}
raw, err := json.Marshal(b.RoomID)
if err != nil {
return fmt.Errorf("build event: room_id: %w", err)
}
obj["room_id"] = raw
return nil
}
// placeRedacts puts Redacts where this room version keeps it: in the
// content from room version 11, at the top level before that.
func (b *Builder) placeRedacts(content json.RawMessage, obj map[string]json.RawMessage, rules roomversion.Rules) (json.RawMessage, error) {
if err := id.ValidateEventID(b.Redacts); err != nil {
return nil, fmt.Errorf("build event: redacts: %w", err)
}
if !rules.RedactsInContent {
raw, err := json.Marshal(b.Redacts)
if err != nil {
return nil, fmt.Errorf("build event: redacts: %w", err)
}
obj["redacts"] = raw
return content, nil
}
var parsed map[string]json.RawMessage
if err := json.Unmarshal(content, &parsed); err != nil {
return nil, fmt.Errorf("build event: content: %w", err)
}
if parsed == nil {
parsed = map[string]json.RawMessage{}
}
raw, err := json.Marshal(b.Redacts)
if err != nil {
return nil, fmt.Errorf("build event: redacts: %w", err)
}
parsed["redacts"] = raw
out, err := json.Marshal(parsed)
if err != nil {
return nil, fmt.Errorf("build event: content: %w", err)
}
return out, nil
}
// orEmpty turns a nil slice into an empty one, so that it encodes as []
// rather than null.
func orEmpty(s []string) []string {
if s == nil {
return []string{}
}
return s
}
+250
View File
@@ -0,0 +1,250 @@
// Package event is the Matrix event (PDU) core: parsing and validating
// events, computing their hashes and identifiers, signing and verifying
// them, and applying the redaction algorithm.
//
// It is pure library code. Nothing here reaches a database or the network,
// and nothing here decides whether an event is *allowed* — that is the
// auth package. The division matters: these are the operations whose
// results every server in a room must agree on byte for byte, so they are
// kept small, deterministic and heavily tested.
//
// An Event is immutable once parsed. The canonical JSON, the event ID and
// the decoded fields are worked out up front and then only read, which is
// what lets an event be passed around and cached without copying.
//
// # A known gap
//
// Room versions 1 to 5 are allowed to contain integers outside the range
// canonical JSON permits, and the spec asks servers to cope where they
// can. Warp does not: such an event is rejected whatever its room version.
// It only arises for events that were already outside the current grammar
// when they were created, and handling them means a second, lenient
// encoder — which can wait until federation (PLAN.md phase 5) makes old
// rooms reachable at all.
package event
import (
"encoding/json"
"errors"
"fmt"
"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
"github.com/dariuszjaniak/warp/server/internal/matrix/id"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
)
// The event types that carry meaning for the event core itself: the ones
// the authorisation rules turn on, plus the two the redaction algorithm
// treats specially.
const (
TypeCreate = "m.room.create"
TypeMember = "m.room.member"
TypeJoinRules = "m.room.join_rules"
TypePowerLevels = "m.room.power_levels"
TypeThirdPartyInvite = "m.room.third_party_invite"
TypeHistoryVisibility = "m.room.history_visibility"
TypeRedaction = "m.room.redaction"
TypeAliases = "m.room.aliases"
)
// The membership values an m.room.member event may carry.
const (
MembershipJoin = "join"
MembershipLeave = "leave"
MembershipInvite = "invite"
MembershipBan = "ban"
MembershipKnock = "knock"
)
// ErrInvalidEvent is the root of every parsing and validation error here,
// so a caller that only needs to answer M_BAD_JSON can test for it.
var ErrInvalidEvent = errors.New("invalid event")
// Event is a parsed, structurally valid PDU.
//
// "Structurally valid" is a lower bar than "allowed in the room": an Event
// has the fields the spec requires, of the right types and within the size
// limits, and its ID has been computed. Whether it may actually be added to
// the room is a question for the auth package.
type Event struct {
rules roomversion.Rules
raw []byte
id string
fields fields
}
// fields are the event properties the server itself reads. Anything else
// stays in raw, untouched, because a server must pass through properties it
// does not understand rather than dropping them.
type fields struct {
RoomID string `json:"room_id"`
Type string `json:"type"`
StateKey *string `json:"state_key"`
Sender string `json:"sender"`
Content json.RawMessage `json:"content"`
PrevEvents []string `json:"prev_events"`
AuthEvents []string `json:"auth_events"`
Depth int64 `json:"depth"`
OriginServerTS int64 `json:"origin_server_ts"`
Hashes hashes `json:"hashes"`
Signatures signing.Signatures `json:"signatures"`
Unsigned json.RawMessage `json:"unsigned"`
// Redacts is the top-level property m.room.redaction used up to room
// version 10. From version 11 the target is content.redacts instead,
// where the redaction algorithm and therefore the signature can cover
// it; Redacts is populated from whichever place this room version uses.
Redacts string `json:"redacts"`
}
type hashes struct {
SHA256 string `json:"sha256"`
}
// Parse validates an event and works out its ID.
//
// raw is canonicalised first, so an event may arrive with its keys in any
// order — what it may not do is use a float, an out-of-range integer or
// invalid UTF-8, because then there are no canonical bytes to hash and no
// two servers would agree on its ID.
func Parse(raw []byte, rules roomversion.Rules) (*Event, error) {
canonical, err := canonicaljson.Encode(raw)
if err != nil {
return nil, fmt.Errorf("%w: %w", ErrInvalidEvent, err)
}
e := &Event{rules: rules, raw: canonical}
if err := json.Unmarshal(canonical, &e.fields); err != nil {
return nil, fmt.Errorf("%w: %w", ErrInvalidEvent, err)
}
if err := e.validate(); err != nil {
return nil, err
}
// From room version 11 the redaction target lives in the content. Read
// it from there so that the rest of the server can ask one question.
if rules.RedactsInContent && e.fields.Type == TypeRedaction {
var content struct {
Redacts string `json:"redacts"`
}
if err := json.Unmarshal(e.fields.Content, &content); err != nil {
return nil, fmt.Errorf("%w: m.room.redaction content: %w", ErrInvalidEvent, err)
}
e.fields.Redacts = content.Redacts
}
if e.id, err = computeEventID(canonical, rules); err != nil {
return nil, err
}
return e, nil
}
// Accessors. They return the decoded fields; the JSON-valued ones hand back
// the event's own storage, which callers must treat as read-only.
// EventID returns the event's identifier, including the '$' sigil.
func (e *Event) EventID() string { return e.id }
// RoomID returns the room this event belongs to. It is empty for an
// m.room.create event in a room version where the room ID *is* the create
// event's ID; use RoomIDOfCreateEvent for that case.
func (e *Event) RoomID() string { return e.fields.RoomID }
// Type returns the event type, such as "m.room.message".
func (e *Event) Type() string { return e.fields.Type }
// Sender returns the user ID that sent the event.
func (e *Event) Sender() string { return e.fields.Sender }
// StateKey returns the state key and whether the event has one. An event
// with a state key is a state event, even if the key is the empty string.
func (e *Event) StateKey() (string, bool) {
if e.fields.StateKey == nil {
return "", false
}
return *e.fields.StateKey, true
}
// IsState reports whether the event is a state event.
func (e *Event) IsState() bool { return e.fields.StateKey != nil }
// Content returns the event content as JSON.
func (e *Event) Content() json.RawMessage { return e.fields.Content }
// PrevEventIDs returns the event IDs of the events this one follows.
func (e *Event) PrevEventIDs() []string { return e.fields.PrevEvents }
// AuthEventIDs returns the event IDs of the events that authorise this one.
func (e *Event) AuthEventIDs() []string { return e.fields.AuthEvents }
// Depth returns the event's depth in the room DAG.
func (e *Event) Depth() int64 { return e.fields.Depth }
// OriginServerTS returns the origin server's timestamp, in milliseconds
// since the epoch. It is a claim by that server, not a fact: it is used for
// ordering and display, never for authorisation.
func (e *Event) OriginServerTS() int64 { return e.fields.OriginServerTS }
// Unsigned returns the unsigned property, which is not covered by the
// event's hashes or signatures and may differ between servers.
func (e *Event) Unsigned() json.RawMessage { return e.fields.Unsigned }
// Signatures returns the event's signature block.
func (e *Event) Signatures() signing.Signatures { return e.fields.Signatures }
// Redacts returns the event ID an m.room.redaction event targets, from
// wherever this room version keeps it. It is empty on any other event.
func (e *Event) Redacts() string { return e.fields.Redacts }
// Rules returns the room version rules the event was parsed under.
func (e *Event) Rules() roomversion.Rules { return e.rules }
// Version returns the event's room version.
func (e *Event) Version() roomversion.Version { return e.rules.Version }
// JSON returns the event as canonical JSON. The slice is the event's own
// and must not be modified.
func (e *Event) JSON() []byte { return e.raw }
// ContentHash returns the content hash carried in the event, as the
// unpadded base64 string it is stored as.
func (e *Event) ContentHash() string { return e.fields.Hashes.SHA256 }
// Membership returns the membership from an m.room.member event's content.
func (e *Event) Membership() (string, error) {
if e.fields.Type != TypeMember {
return "", fmt.Errorf("%s is not an m.room.member event", e.id)
}
var content struct {
Membership string `json:"membership"`
}
if err := json.Unmarshal(e.fields.Content, &content); err != nil {
return "", fmt.Errorf("%w: m.room.member content: %w", ErrInvalidEvent, err)
}
return content.Membership, nil
}
// SenderServer returns the server name of the event's sender, which is the
// server that must have signed it.
func (e *Event) SenderServer() (string, error) {
return id.ServerNameOf(e.fields.Sender)
}
// RoomIDOfCreateEvent returns the room ID an m.room.create event creates,
// in the room versions where the room ID is the create event's own ID with
// a '!' in place of the '$'.
//
// Deriving the room ID this way is what removed the domain from room IDs:
// a room is no longer named after the server that happened to create it.
func RoomIDOfCreateEvent(e *Event) (string, error) {
if e.fields.Type != TypeCreate {
return "", fmt.Errorf("%s is not an m.room.create event", e.id)
}
if !e.rules.RoomIDIsCreateEventID {
// In these versions the create event carries the room ID like any
// other event, so there is nothing to derive.
return e.fields.RoomID, nil
}
return "!" + e.id[1:], nil
}
+945
View File
@@ -0,0 +1,945 @@
package event
import (
"crypto/ed25519"
"encoding/base64"
"encoding/json"
"errors"
"strings"
"testing"
"time"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
)
// The signing key from the spec appendices, section "Cryptographic Test
// Vectors".
const specSeed = "YJDBA9Xnr2sVqXD9Vj7XVUnmFZcZrlw8Md7kMW+3XA1"
func specKey(t *testing.T) *signing.KeyPair {
t.Helper()
seed, err := base64.RawStdEncoding.DecodeString(specSeed)
if err != nil {
t.Fatalf("decode the spec seed: %v", err)
}
priv := ed25519.NewKeyFromSeed(seed)
pub, ok := priv.Public().(ed25519.PublicKey)
if !ok {
t.Fatal("unexpected public key type")
}
return &signing.KeyPair{ID: "1", Private: priv, Public: pub}
}
func testKey(t *testing.T) *signing.KeyPair {
t.Helper()
key, err := signing.Generate("test")
if err != nil {
t.Fatalf("generate a signing key: %v", err)
}
return key
}
// TestSpecEventSigningVectors runs the event signing vectors from the spec
// appendices. They were produced for the original event format, so they go
// through a room version whose redaction rules still protect the top-level
// origin and event_id properties — the point of the test is the hashing
// and signing, which has not changed since.
func TestSpecEventSigningVectors(t *testing.T) {
key := specKey(t)
rules := roomversion.MustGet(roomversion.V3)
cases := []struct {
name string
in string
want string
}{
{
name: "minimal event",
in: `{
"room_id": "!x:domain",
"sender": "@a:domain",
"origin": "domain",
"origin_server_ts": 1000000,
"signatures": {},
"hashes": {},
"type": "X",
"content": {},
"prev_events": [],
"auth_events": [],
"depth": 3,
"unsigned": {"age_ts": 1000000}
}`,
want: `{"auth_events":[],"content":{},"depth":3,` +
`"hashes":{"sha256":"5jM4wQpv6lnBo7CLIghJuHdW+s2CMBJPUOGOC89ncos"},` +
`"origin":"domain","origin_server_ts":1000000,"prev_events":[],` +
`"room_id":"!x:domain","sender":"@a:domain","signatures":{"domain":{"ed25519:1":` +
`"KxwGjPSDEtvnFgU00fwFz+l6d2pJM6XBIaMEn81SXPTRl16AqLAYqfIReFGZlHi5KLjAWbOoMszkwsQma+lYAg"}},` +
`"type":"X","unsigned":{"age_ts":1000000}}`,
},
{
name: "event with redactable content",
in: `{
"content": {"body": "Here is the message content"},
"event_id": "$0:domain",
"origin": "domain",
"origin_server_ts": 1000000,
"type": "m.room.message",
"room_id": "!r:domain",
"sender": "@u:domain",
"signatures": {},
"unsigned": {"age_ts": 1000000}
}`,
want: `{"content":{"body":"Here is the message content"},"event_id":"$0:domain",` +
`"hashes":{"sha256":"onLKD1bGljeBWQhWZ1kaP9SorVmRQNdN5aM2JYU2n/g"},` +
`"origin":"domain","origin_server_ts":1000000,"room_id":"!r:domain",` +
`"sender":"@u:domain","signatures":{"domain":{"ed25519:1":` +
`"Wm+VzmOUOz08Ds+0NTWb1d4CZrVsJSikkeRxh6aCcUwu6pNC78FunoD7KNWzqFn241eYHYMGCA5McEiVPdhzBA"}},` +
`"type":"m.room.message","unsigned":{"age_ts":1000000}}`,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, err := hashAndSign([]byte(tc.in), rules, "domain", key)
if err != nil {
t.Fatalf("hashAndSign: %v", err)
}
if string(got) != tc.want {
t.Errorf("hashAndSign() =\n %s\nwant\n %s", got, tc.want)
}
})
}
}
// buildEvent is the shorthand the tests below use for a signed event in a
// given room version.
func buildEvent(t *testing.T, rules roomversion.Rules, key *signing.KeyPair, b Builder) *Event {
t.Helper()
if b.Sender == "" {
b.Sender = "@alice:warp.test"
}
// A version 12 create event has no room ID: the room ID comes from it.
derivesRoomID := rules.RoomIDIsCreateEventID && b.Type == TypeCreate
if b.RoomID == "" && !derivesRoomID {
b.RoomID = "!room:warp.test"
}
e, err := b.Build(time.UnixMilli(1700000000000), key, rules)
if err != nil {
t.Fatalf("Build %s: %v", b.Type, err)
}
return e
}
func TestBuildAndParse(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{
Type: "m.room.message",
Content: json.RawMessage(`{"body":"hello","msgtype":"m.text"}`),
PrevEvents: []string{"$abc"},
AuthEvents: []string{"$create"},
Depth: 7,
})
if !strings.HasPrefix(e.EventID(), "$") {
t.Errorf("event ID %q does not start with $", e.EventID())
}
if e.Type() != "m.room.message" {
t.Errorf("Type() = %q", e.Type())
}
if e.Sender() != "@alice:warp.test" {
t.Errorf("Sender() = %q", e.Sender())
}
if e.RoomID() != "!room:warp.test" {
t.Errorf("RoomID() = %q", e.RoomID())
}
if e.Depth() != 7 {
t.Errorf("Depth() = %d", e.Depth())
}
if e.OriginServerTS() != 1700000000000 {
t.Errorf("OriginServerTS() = %d", e.OriginServerTS())
}
if _, ok := e.StateKey(); ok {
t.Error("a message event should have no state key")
}
if e.IsState() {
t.Error("IsState() = true for a message event")
}
if err := e.VerifyContentHash(); err != nil {
t.Errorf("VerifyContentHash: %v", err)
}
if err := e.VerifySignature("warp.test", key.KeyID(), key.Public); err != nil {
t.Errorf("VerifySignature: %v", err)
}
// Re-parsing the event's own JSON must produce the same ID, which is
// what makes it safe to store events as JSON and parse them back.
again, err := Parse(e.JSON(), rules)
if err != nil {
t.Fatalf("re-parse: %v", err)
}
if again.EventID() != e.EventID() {
t.Errorf("re-parsed event ID = %q, want %q", again.EventID(), e.EventID())
}
}
func TestBuildStateEventWithEmptyStateKey(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
empty := ""
e := buildEvent(t, rules, key, Builder{
Type: TypePowerLevels,
StateKey: &empty,
Content: json.RawMessage(`{"users":{"@alice:warp.test":100}}`),
})
sk, ok := e.StateKey()
if !ok {
t.Fatal("StateKey() reports no state key")
}
if sk != "" {
t.Errorf("StateKey() = %q, want the empty string", sk)
}
if !e.IsState() {
t.Error("IsState() = false for a power levels event")
}
}
// TestEventIDFormatByVersion checks the two encodings of the reference
// hash. Room version 3 uses standard base64 and 4 onwards URL-safe, which
// is why the same event has two different IDs in the two rooms.
func TestEventIDFormatByVersion(t *testing.T) {
key := specKey(t)
body := Builder{
Type: "m.room.message",
Sender: "@a:domain",
RoomID: "!r:domain",
Content: json.RawMessage(`{"body":"ÿþýü"}`),
}
ids := map[roomversion.Version]string{}
for _, v := range []roomversion.Version{roomversion.V3, roomversion.V4} {
e, err := body.Build(time.UnixMilli(1000000), key, roomversion.MustGet(v))
if err != nil {
t.Fatalf("Build for version %s: %v", v, err)
}
ids[v] = e.EventID()
}
// Both encode the same 32-byte hash, so the IDs are the same length
// and differ only where the alphabets differ.
if len(ids[roomversion.V3]) != len(ids[roomversion.V4]) {
t.Errorf("IDs have different lengths: %q and %q", ids[roomversion.V3], ids[roomversion.V4])
}
std := strings.TrimPrefix(ids[roomversion.V3], "$")
urlSafe := strings.TrimPrefix(ids[roomversion.V4], "$")
if decoded, err := base64.RawStdEncoding.DecodeString(std); err != nil || len(decoded) != 32 {
t.Errorf("version 3 ID is not 32 bytes of standard base64: %q (%v)", std, err)
}
if decoded, err := base64.RawURLEncoding.DecodeString(urlSafe); err != nil || len(decoded) != 32 {
t.Errorf("version 4 ID is not 32 bytes of URL-safe base64: %q (%v)", urlSafe, err)
}
if strings.ContainsAny(urlSafe, "+/") {
t.Errorf("version 4 ID contains a character that is not URL-safe: %q", urlSafe)
}
}
// TestRedactionKeepsEventIDAndSignature is the property the whole hashing
// scheme exists for: a redacted event is still provably the event it claims
// to be.
func TestRedactionKeepsEventIDAndSignature(t *testing.T) {
key := testKey(t)
for _, v := range []roomversion.Version{roomversion.V3, roomversion.V10, roomversion.V11, roomversion.V12} {
t.Run(string(v), func(t *testing.T) {
rules := roomversion.MustGet(v)
e := buildEvent(t, rules, key, Builder{
Type: "m.room.message",
Content: json.RawMessage(`{"body":"something private","msgtype":"m.text"}`),
})
redacted, err := e.Redact()
if err != nil {
t.Fatalf("Redact: %v", err)
}
if redacted.EventID() != e.EventID() {
t.Errorf("redaction changed the event ID: %q became %q", e.EventID(), redacted.EventID())
}
if err := redacted.VerifySignature("warp.test", key.KeyID(), key.Public); err != nil {
t.Errorf("the signature on the redacted event does not verify: %v", err)
}
if string(redacted.Content()) != "{}" {
t.Errorf("redacted content = %s, want {}", redacted.Content())
}
// The content hash no longer matches, which is exactly how a
// receiver tells a redacted copy from a tampered one.
if err := redacted.VerifyContentHash(); !errors.Is(err, ErrContentHashMismatch) {
t.Errorf("VerifyContentHash on a redacted event = %v, want ErrContentHashMismatch", err)
}
})
}
}
// TestRedactionProtectedKeys walks the keys each room version protects.
func TestRedactionProtectedKeys(t *testing.T) {
cases := []struct {
name string
version roomversion.Version
event string
wantKeys []string
wantContent string
}{
{
name: "version 10 protects the legacy top-level keys",
version: roomversion.V10,
event: `{"type":"m.room.message","room_id":"!r:domain","sender":"@u:domain",
"content":{"body":"x"},"origin":"domain","membership":"join","prev_state":[],
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{},"age_ts":5}`,
wantKeys: []string{
"content", "hashes", "membership", "origin", "origin_server_ts",
"prev_state", "room_id", "sender", "signatures", "type",
},
wantContent: `{}`,
},
{
name: "version 11 drops the legacy top-level keys",
version: roomversion.V11,
event: `{"type":"m.room.message","room_id":"!r:domain","sender":"@u:domain",
"content":{"body":"x"},"origin":"domain","membership":"join","prev_state":[],
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{},"age_ts":5}`,
wantKeys: []string{
"content", "hashes", "origin_server_ts", "room_id",
"sender", "signatures", "type",
},
wantContent: `{}`,
},
{
name: "version 10 keeps only the creator of a create event",
version: roomversion.V10,
event: `{"type":"m.room.create","room_id":"!r:domain","sender":"@u:domain",
"content":{"creator":"@u:domain","room_version":"10","m.federate":false},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"creator":"@u:domain"}`,
},
{
name: "version 11 keeps all of a create event's content",
version: roomversion.V11,
event: `{"type":"m.room.create","room_id":"!r:domain","sender":"@u:domain",
"content":{"room_version":"11","m.federate":false},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"m.federate":false,"room_version":"11"}`,
},
{
name: "version 8 keeps the join rule and allow list",
version: roomversion.V8,
event: `{"type":"m.room.join_rules","room_id":"!r:domain","sender":"@u:domain",
"content":{"join_rule":"restricted","allow":[{"type":"m.room_membership"}],"extra":1},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"allow":[{"type":"m.room_membership"}],"join_rule":"restricted"}`,
},
{
name: "version 7 keeps only the join rule",
version: roomversion.V7,
event: `{"type":"m.room.join_rules","room_id":"!r:domain","sender":"@u:domain",
"content":{"join_rule":"restricted","allow":[{"type":"m.room_membership"}]},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"join_rule":"restricted"}`,
},
{
name: "version 8 redacts the restricted join authorisation",
version: roomversion.V8,
event: `{"type":"m.room.member","room_id":"!r:domain","sender":"@u:domain","state_key":"@u:domain",
"content":{"membership":"join","join_authorised_via_users_server":"@mod:other"},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"membership":"join"}`,
},
{
name: "version 9 keeps the restricted join authorisation",
version: roomversion.V9,
event: `{"type":"m.room.member","room_id":"!r:domain","sender":"@u:domain","state_key":"@u:domain",
"content":{"membership":"join","join_authorised_via_users_server":"@mod:other","displayname":"U"},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"join_authorised_via_users_server":"@mod:other","membership":"join"}`,
},
{
name: "version 10 drops a third-party invite entirely",
version: roomversion.V10,
event: `{"type":"m.room.member","room_id":"!r:domain","sender":"@u:domain","state_key":"@u:domain",
"content":{"membership":"invite","third_party_invite":{"display_name":"d","signed":{"token":"t"}}},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"membership":"invite"}`,
},
{
name: "version 11 keeps the signed part of a third-party invite",
version: roomversion.V11,
event: `{"type":"m.room.member","room_id":"!r:domain","sender":"@u:domain","state_key":"@u:domain",
"content":{"membership":"invite","third_party_invite":{"display_name":"d","signed":{"token":"t"}}},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"membership":"invite","third_party_invite":{"signed":{"token":"t"}}}`,
},
{
name: "version 10 loses the target of a redaction",
version: roomversion.V10,
event: `{"type":"m.room.redaction","room_id":"!r:domain","sender":"@u:domain",
"content":{"reason":"spam"},"redacts":"$target",
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantKeys: []string{"content", "hashes", "origin_server_ts", "room_id", "sender", "signatures", "type"},
wantContent: `{}`,
},
{
name: "version 11 keeps the target of a redaction",
version: roomversion.V11,
event: `{"type":"m.room.redaction","room_id":"!r:domain","sender":"@u:domain",
"content":{"redacts":"$target","reason":"spam"},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"redacts":"$target"}`,
},
{
name: "version 11 keeps the invite level of a power levels event",
version: roomversion.V11,
event: `{"type":"m.room.power_levels","room_id":"!r:domain","sender":"@u:domain","state_key":"",
"content":{"invite":50,"ban":50,"notifications":{"room":20}},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"ban":50,"invite":50}`,
},
{
name: "version 10 drops the invite level of a power levels event",
version: roomversion.V10,
event: `{"type":"m.room.power_levels","room_id":"!r:domain","sender":"@u:domain","state_key":"",
"content":{"invite":50,"ban":50,"notifications":{"room":20}},
"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{"ban":50}`,
},
{
name: "an unknown type keeps no content at all",
version: roomversion.V11,
event: `{"type":"com.example.thing","room_id":"!r:domain","sender":"@u:domain",
"content":{"anything":1},"origin_server_ts":1,"hashes":{"sha256":"x"},"signatures":{}}`,
wantContent: `{}`,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
rules := roomversion.MustGet(tc.version)
got, err := redactJSON([]byte(tc.event), rules)
if err != nil {
t.Fatalf("redactJSON: %v", err)
}
var obj map[string]json.RawMessage
if err := json.Unmarshal(got, &obj); err != nil {
t.Fatalf("the redacted event is not an object: %v", err)
}
if tc.wantKeys != nil {
var keys []string
for k := range obj {
keys = append(keys, k)
}
if got, want := joinSorted(keys), strings.Join(tc.wantKeys, ","); got != want {
t.Errorf("top-level keys = %s, want %s", got, want)
}
}
if content := string(obj["content"]); content != tc.wantContent {
t.Errorf("content = %s, want %s", content, tc.wantContent)
}
})
}
}
func joinSorted(keys []string) string {
// canonicaljson sorts object keys, so the redacted event already has
// them in order; this is only for a readable failure message.
out := make([]string, len(keys))
copy(out, keys)
for i := range out {
for j := i + 1; j < len(out); j++ {
if out[j] < out[i] {
out[i], out[j] = out[j], out[i]
}
}
}
return strings.Join(out, ",")
}
// TestRedactsLocation checks that the builder puts a redaction's target
// where the room version keeps it, and that Redacts reads it back from
// either place.
func TestRedactsLocation(t *testing.T) {
key := testKey(t)
t.Run("version 10 uses the top level", func(t *testing.T) {
rules := roomversion.MustGet(roomversion.V10)
e := buildEvent(t, rules, key, Builder{
Type: TypeRedaction,
Redacts: "$target",
Content: json.RawMessage(`{"reason":"spam"}`),
})
if e.Redacts() != "$target" {
t.Errorf("Redacts() = %q, want $target", e.Redacts())
}
if !strings.Contains(string(e.JSON()), `"redacts":"$target"`) {
t.Errorf("redacts is missing from the event: %s", e.JSON())
}
if strings.Contains(string(e.Content()), "redacts") {
t.Errorf("redacts should not be in the content: %s", e.Content())
}
})
t.Run("version 11 uses the content", func(t *testing.T) {
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{
Type: TypeRedaction,
Redacts: "$target",
Content: json.RawMessage(`{"reason":"spam"}`),
})
if e.Redacts() != "$target" {
t.Errorf("Redacts() = %q, want $target", e.Redacts())
}
if !strings.Contains(string(e.Content()), `"redacts":"$target"`) {
t.Errorf("redacts is missing from the content: %s", e.Content())
}
})
}
// TestCreateEventRoomIDByVersion covers the room version 12 change that
// makes the room ID the create event's own ID.
func TestCreateEventRoomIDByVersion(t *testing.T) {
key := testKey(t)
t.Run("version 11 carries a room ID", func(t *testing.T) {
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{
Type: TypeCreate,
Content: json.RawMessage(`{"room_version":"11"}`),
})
if e.RoomID() != "!room:warp.test" {
t.Errorf("RoomID() = %q", e.RoomID())
}
roomID, err := RoomIDOfCreateEvent(e)
if err != nil {
t.Fatalf("RoomIDOfCreateEvent: %v", err)
}
if roomID != "!room:warp.test" {
t.Errorf("RoomIDOfCreateEvent() = %q", roomID)
}
})
t.Run("version 12 derives the room ID", func(t *testing.T) {
rules := roomversion.MustGet(roomversion.V12)
e := buildEvent(t, rules, key, Builder{
Type: TypeCreate,
Content: json.RawMessage(`{"room_version":"12"}`),
})
if e.RoomID() != "" {
t.Errorf("RoomID() = %q, want it to be absent", e.RoomID())
}
roomID, err := RoomIDOfCreateEvent(e)
if err != nil {
t.Fatalf("RoomIDOfCreateEvent: %v", err)
}
if want := "!" + strings.TrimPrefix(e.EventID(), "$"); roomID != want {
t.Errorf("RoomIDOfCreateEvent() = %q, want %q", roomID, want)
}
if strings.Contains(roomID, ":") {
t.Errorf("a version 12 room ID should carry no domain: %q", roomID)
}
})
t.Run("version 12 rejects a create event with a room ID", func(t *testing.T) {
rules := roomversion.MustGet(roomversion.V12)
b := Builder{
Type: TypeCreate,
Sender: "@alice:warp.test",
RoomID: "!room:warp.test",
Content: json.RawMessage(`{"room_version":"12"}`),
}
if _, err := b.Build(time.UnixMilli(1), key, rules); err == nil {
t.Error("Build accepted a room_id on a version 12 create event")
}
})
}
func TestVerifyContentHashDetectsTampering(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{
Type: "m.room.message",
Content: json.RawMessage(`{"body":"the original","msgtype":"m.text"}`),
})
tampered := strings.Replace(string(e.JSON()), "the original", "a forgery!!!", 1)
if tampered == string(e.JSON()) {
t.Fatal("the test did not change anything")
}
parsed, err := Parse([]byte(tampered), rules)
if err != nil {
t.Fatalf("Parse: %v", err)
}
if err := parsed.VerifyContentHash(); !errors.Is(err, ErrContentHashMismatch) {
t.Errorf("VerifyContentHash = %v, want ErrContentHashMismatch", err)
}
// The signature still verifies, and that is not a hole: it covers the
// redacted event, whose content is {} either way. Rewriting content is
// therefore indistinguishable from being sent a redacted copy, and the
// receiver's answer to a failed hash check is to use the redacted form
// — so the rewritten content never reaches anybody.
if err := parsed.VerifySignature("warp.test", key.KeyID(), key.Public); err != nil {
t.Errorf("the signature should still cover the redacted event: %v", err)
}
// Rewriting the hash to match the new content is what the signature
// does catch, because the signature covers hashes.
sum, err := contentHash(parsed.JSON())
if err != nil {
t.Fatal(err)
}
reHashed := strings.Replace(string(parsed.JSON()),
`"sha256":"`+parsed.ContentHash()+`"`,
`"sha256":"`+base64.RawStdEncoding.EncodeToString(sum[:])+`"`, 1)
forged, err := Parse([]byte(reHashed), rules)
if err != nil {
t.Fatalf("Parse: %v", err)
}
if err := forged.VerifyContentHash(); err != nil {
t.Fatalf("the test did not recompute the hash correctly: %v", err)
}
if err := forged.VerifySignature("warp.test", key.KeyID(), key.Public); err == nil {
t.Error("the signature verifies on an event whose content and hash were both rewritten")
}
if forged.EventID() == e.EventID() {
t.Error("rewriting the hash should change the event ID")
}
}
func TestSignAddsASecondSignature(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{
Type: "m.room.member",
Content: json.RawMessage(`{"membership":"invite"}`),
})
other, err := signing.Generate("other")
if err != nil {
t.Fatal(err)
}
countersigned, err := e.Sign("other.example", other)
if err != nil {
t.Fatalf("Sign: %v", err)
}
if countersigned.EventID() != e.EventID() {
t.Errorf("countersigning changed the event ID: %q became %q", e.EventID(), countersigned.EventID())
}
if err := countersigned.VerifySignature("warp.test", key.KeyID(), key.Public); err != nil {
t.Errorf("the original signature no longer verifies: %v", err)
}
if err := countersigned.VerifySignature("other.example", other.KeyID(), other.Public); err != nil {
t.Errorf("the second signature does not verify: %v", err)
}
}
func TestParseRejects(t *testing.T) {
rules := roomversion.MustGet(roomversion.V11)
const sigs = `"hashes":{"sha256":"x"},"signatures":{"d":{"ed25519:1":"s"}}`
cases := []struct {
name string
in string
want string
}{
{"no type", `{"room_id":"!r:d","sender":"@u:d","content":{},` + sigs + `}`, "type is required"},
{"no sender", `{"type":"m.x","room_id":"!r:d","content":{},` + sigs + `}`, "sender"},
{"sender is not a user ID", `{"type":"m.x","room_id":"!r:d","sender":"alice","content":{},` + sigs + `}`, "sender"},
{"no room ID", `{"type":"m.x","sender":"@u:d","content":{},` + sigs + `}`, "room_id"},
{"room ID with the wrong sigil", `{"type":"m.x","room_id":"@r:d","sender":"@u:d","content":{},` + sigs + `}`, "room_id"},
{"no content", `{"type":"m.x","room_id":"!r:d","sender":"@u:d",` + sigs + `}`, "content is required"},
{"content is not an object", `{"type":"m.x","room_id":"!r:d","sender":"@u:d","content":[],` + sigs + `}`, "content must be an object"},
{"content is null", `{"type":"m.x","room_id":"!r:d","sender":"@u:d","content":null,` + sigs + `}`, "content must be an object"},
{"no content hash", `{"type":"m.x","room_id":"!r:d","sender":"@u:d","content":{},"signatures":{"d":{"e:1":"s"}}}`, "hashes.sha256 is required"},
{"no signatures", `{"type":"m.x","room_id":"!r:d","sender":"@u:d","content":{},"hashes":{"sha256":"x"}}`, "signatures is required"},
{"a float in the content", `{"type":"m.x","room_id":"!r:d","sender":"@u:d","content":{"a":1.5},` + sigs + `}`, "floats"},
{"a prev event that is not an event ID", `{"type":"m.x","room_id":"!r:d","sender":"@u:d","content":{},"prev_events":["abc"],` + sigs + `}`, "prev_events"},
{"depth above the limit", `{"type":"m.x","room_id":"!r:d","sender":"@u:d","content":{},"depth":9007199254740992,` + sigs + `}`, "outside the allowed range"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
_, err := Parse([]byte(tc.in), rules)
if err == nil {
t.Fatal("Parse succeeded, want an error")
}
if !strings.Contains(err.Error(), tc.want) {
t.Errorf("error = %q, want it to mention %q", err, tc.want)
}
})
}
}
func TestParseRejectsOversizedEvent(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
b := Builder{
Type: "m.room.message",
Sender: "@alice:warp.test",
RoomID: "!room:warp.test",
Content: json.RawMessage(`{"body":"` + strings.Repeat("x", MaxEventSize) + `"}`),
}
_, err := b.Build(time.UnixMilli(1), key, rules)
if err == nil {
t.Fatal("Build accepted an event over the size limit")
}
if !strings.Contains(err.Error(), "the limit is") {
t.Errorf("error = %q, want it to mention the limit", err)
}
}
func TestParseRejectsTooManyDAGReferences(t *testing.T) {
rules := roomversion.MustGet(roomversion.V11)
ids := make([]string, MaxPrevEvents+1)
for i := range ids {
ids[i] = "$event" + string(rune('a'+i))
}
encoded, err := json.Marshal(ids)
if err != nil {
t.Fatal(err)
}
raw := `{"type":"m.x","room_id":"!r:d","sender":"@u:d","content":{},"prev_events":` +
string(encoded) + `,"hashes":{"sha256":"x"},"signatures":{"d":{"ed25519:1":"s"}}}`
if _, err := Parse([]byte(raw), rules); err == nil {
t.Error("Parse accepted more than the maximum number of prev_events")
}
}
func TestMembership(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
target := "@bob:warp.test"
e := buildEvent(t, rules, key, Builder{
Type: TypeMember,
StateKey: &target,
Content: json.RawMessage(`{"membership":"invite"}`),
})
got, err := e.Membership()
if err != nil {
t.Fatalf("Membership: %v", err)
}
if got != MembershipInvite {
t.Errorf("Membership() = %q, want %q", got, MembershipInvite)
}
other := buildEvent(t, rules, key, Builder{Type: "m.room.message", Content: json.RawMessage(`{}`)})
if _, err := other.Membership(); err == nil {
t.Error("Membership on a message event succeeded, want an error")
}
}
func TestSenderServer(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{Type: "m.room.message", Content: json.RawMessage(`{}`)})
got, err := e.SenderServer()
if err != nil {
t.Fatalf("SenderServer: %v", err)
}
if got != "warp.test" {
t.Errorf("SenderServer() = %q, want warp.test", got)
}
}
func TestBuilderRejects(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
cases := []struct {
name string
b Builder
}{
{"no sender", Builder{Type: "m.x", RoomID: "!r:d"}},
{"sender is not a user ID", Builder{Type: "m.x", RoomID: "!r:d", Sender: "alice"}},
{"no type", Builder{RoomID: "!r:d", Sender: "@u:d"}},
{"no room ID", Builder{Type: "m.x", Sender: "@u:d"}},
{"negative depth", Builder{Type: "m.x", RoomID: "!r:d", Sender: "@u:d", Depth: -1}},
{"redacts is not an event ID", Builder{Type: TypeRedaction, RoomID: "!r:d", Sender: "@u:d", Redacts: "nope"}},
{"content is not an object", Builder{Type: "m.x", RoomID: "!r:d", Sender: "@u:d", Content: json.RawMessage(`[]`)}},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if _, err := tc.b.Build(time.UnixMilli(1), key, rules); err == nil {
t.Error("Build succeeded, want an error")
}
})
}
}
// TestUnsignedIsNotCoveredBySignature mirrors the federation case where a
// server rewrites unsigned.age in transit.
func TestUnsignedIsNotCoveredBySignature(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{
Type: "m.room.message",
Content: json.RawMessage(`{"body":"hi"}`),
Unsigned: json.RawMessage(`{"age":10}`),
})
rewritten := strings.Replace(string(e.JSON()), `"age":10`, `"age":99`, 1)
parsed, err := Parse([]byte(rewritten), rules)
if err != nil {
t.Fatalf("Parse: %v", err)
}
if parsed.EventID() != e.EventID() {
t.Errorf("rewriting unsigned changed the event ID: %q became %q", e.EventID(), parsed.EventID())
}
if err := parsed.VerifySignature("warp.test", key.KeyID(), key.Public); err != nil {
t.Errorf("rewriting unsigned broke the signature: %v", err)
}
if err := parsed.VerifyContentHash(); err != nil {
t.Errorf("rewriting unsigned broke the content hash: %v", err)
}
}
// FuzzParse checks that no input makes Parse panic, and that any event it
// accepts round-trips through its own JSON to the same ID.
func FuzzParse(f *testing.F) {
rules := roomversion.MustGet(roomversion.V11)
f.Add([]byte(`{"type":"m.x","room_id":"!r:d","sender":"@u:d","content":{},` +
`"hashes":{"sha256":"x"},"signatures":{"d":{"ed25519:1":"s"}}}`))
f.Add([]byte(`{}`))
f.Fuzz(func(t *testing.T, in []byte) {
e, err := Parse(in, rules)
if err != nil {
return
}
again, err := Parse(e.JSON(), rules)
if err != nil {
t.Fatalf("re-parsing an accepted event failed: %v\nevent: %s", err, e.JSON())
}
if again.EventID() != e.EventID() {
t.Fatalf("event ID is not stable: %q then %q\nevent: %s", e.EventID(), again.EventID(), e.JSON())
}
if _, err := e.Redact(); err != nil {
t.Fatalf("redacting an accepted event failed: %v\nevent: %s", err, e.JSON())
}
})
}
// TestAccessors walks the plain field accessors, which the rest of the
// server reads an event through.
func TestAccessors(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{
Type: "m.room.message",
Content: json.RawMessage(`{"body":"hi"}`),
PrevEvents: []string{"$prev"},
AuthEvents: []string{"$auth"},
Depth: 4,
Unsigned: json.RawMessage(`{"age":7}`),
})
if got := e.PrevEventIDs(); len(got) != 1 || got[0] != "$prev" {
t.Errorf("PrevEventIDs() = %v", got)
}
if got := e.AuthEventIDs(); len(got) != 1 || got[0] != "$auth" {
t.Errorf("AuthEventIDs() = %v", got)
}
if got := string(e.Unsigned()); got != `{"age":7}` {
t.Errorf("Unsigned() = %s", got)
}
if _, ok := e.Signatures().Get("warp.test", key.KeyID()); !ok {
t.Errorf("Signatures() has no entry for warp.test: %v", e.Signatures())
}
if e.Rules().Version != roomversion.V11 {
t.Errorf("Rules().Version = %q", e.Rules().Version)
}
if e.Version() != roomversion.V11 {
t.Errorf("Version() = %q", e.Version())
}
if e.ContentHash() == "" {
t.Error("ContentHash() is empty")
}
// The reference hash is what the event ID is made of, so it must agree
// with the ID the event already has.
hash, err := e.ReferenceHash()
if err != nil {
t.Fatalf("ReferenceHash: %v", err)
}
if len(hash) != 32 {
t.Errorf("ReferenceHash() is %d bytes, want 32", len(hash))
}
if want := "$" + base64.RawURLEncoding.EncodeToString(hash); want != e.EventID() {
t.Errorf("the reference hash encodes to %q, but the event ID is %q", want, e.EventID())
}
}
// TestContentHashAcceptsPaddedBase64 covers the leniency the spec asks for
// when decoding: values get copied between implementations by hand, so a
// padded hash still has to verify.
func TestContentHashAcceptsPaddedBase64(t *testing.T) {
key := testKey(t)
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{Type: "m.room.message", Content: json.RawMessage(`{"body":"hi"}`)})
sum, err := contentHash(e.JSON())
if err != nil {
t.Fatal(err)
}
padded := strings.Replace(string(e.JSON()),
`"sha256":"`+e.ContentHash()+`"`,
`"sha256":"`+base64.StdEncoding.EncodeToString(sum[:])+`"`, 1)
parsed, err := Parse([]byte(padded), rules)
if err != nil {
t.Fatalf("Parse: %v", err)
}
if err := parsed.VerifyContentHash(); err != nil {
t.Errorf("a padded content hash did not verify: %v", err)
}
}
// TestRedactARedaction checks that the target of a redaction survives
// being redacted itself, which is why the property moved into the content.
func TestRedactARedaction(t *testing.T) {
key := testKey(t)
t.Run("version 11 keeps it", func(t *testing.T) {
rules := roomversion.MustGet(roomversion.V11)
e := buildEvent(t, rules, key, Builder{
Type: TypeRedaction,
Redacts: "$target",
Content: json.RawMessage(`{"reason":"spam"}`),
})
redacted, err := e.Redact()
if err != nil {
t.Fatalf("Redact: %v", err)
}
if redacted.Redacts() != "$target" {
t.Errorf("Redacts() = %q after redaction, want $target", redacted.Redacts())
}
})
t.Run("version 10 loses it", func(t *testing.T) {
rules := roomversion.MustGet(roomversion.V10)
e := buildEvent(t, rules, key, Builder{
Type: TypeRedaction,
Redacts: "$target",
Content: json.RawMessage(`{"reason":"spam"}`),
})
redacted, err := e.Redact()
if err != nil {
t.Fatalf("Redact: %v", err)
}
if redacted.Redacts() != "" {
t.Errorf("Redacts() = %q after redaction, want it to be gone", redacted.Redacts())
}
})
}
+136
View File
@@ -0,0 +1,136 @@
package event
import (
"bytes"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
)
// ErrContentHashMismatch means an event's content does not match the hash
// it carries.
//
// Over federation this is not necessarily an attack and not necessarily
// fatal: the usual cause is being sent a redacted copy of an event whose
// signature still verifies, because the signature covers only the keys
// redaction keeps. The caller's answer is to use the redacted form rather
// than to drop the event.
var ErrContentHashMismatch = errors.New("event content does not match its content hash")
// contentHash computes the content hash of an event: SHA-256 over the
// canonical JSON of the event with unsigned, signatures and hashes removed.
//
// Each of the three is excluded for its own reason. unsigned is explicitly
// rewritable in transit; signatures depend on the hash, so including it
// would be circular; and hashes is left out so that a future algorithm can
// be added alongside sha256 without changing what sha256 covers.
func contentHash(raw []byte) ([sha256.Size]byte, error) {
obj, err := decodeObject(raw)
if err != nil {
return [sha256.Size]byte{}, err
}
delete(obj, "unsigned")
delete(obj, "signatures")
delete(obj, "hashes")
encoded, err := canonicaljson.Marshal(obj)
if err != nil {
return [sha256.Size]byte{}, fmt.Errorf("%w: content hash: %w", ErrInvalidEvent, err)
}
return sha256.Sum256(encoded), nil
}
// referenceHash computes the reference hash of an event, which is what its
// ID is made of: the event is redacted, then signatures and unsigned are
// removed, then it is canonicalised and hashed.
//
// Redacting first is what makes an event ID survive redaction — the ID
// covers only the keys a redaction keeps, so redacting an event does not
// rename it.
func referenceHash(raw []byte, rules roomversion.Rules) ([sha256.Size]byte, error) {
redacted, err := redactJSON(raw, rules)
if err != nil {
return [sha256.Size]byte{}, err
}
obj, err := decodeObject(redacted)
if err != nil {
return [sha256.Size]byte{}, err
}
delete(obj, "signatures")
delete(obj, "unsigned")
encoded, err := canonicaljson.Marshal(obj)
if err != nil {
return [sha256.Size]byte{}, fmt.Errorf("%w: reference hash: %w", ErrInvalidEvent, err)
}
return sha256.Sum256(encoded), nil
}
// computeEventID derives an event ID from the event's reference hash.
func computeEventID(raw []byte, rules roomversion.Rules) (string, error) {
sum, err := referenceHash(raw, rules)
if err != nil {
return "", err
}
switch rules.EventIDFormat {
case roomversion.EventIDFormatSHA256:
return "$" + base64.RawStdEncoding.EncodeToString(sum[:]), nil
case roomversion.EventIDFormatSHA256URLSafe:
// URL-safe, because an ID with a '/' in it cannot be put in a path
// — which is where event IDs ended up in the federation API.
return "$" + base64.RawURLEncoding.EncodeToString(sum[:]), nil
default:
return "", fmt.Errorf("%w: room version %s does not derive event IDs from a hash",
ErrInvalidEvent, rules.Version)
}
}
// ReferenceHash returns the event's reference hash, unencoded. Federation
// needs it for the room version 1 and 2 event format and for auth chain
// comparisons.
func (e *Event) ReferenceHash() ([]byte, error) {
sum, err := referenceHash(e.raw, e.rules)
if err != nil {
return nil, err
}
return sum[:], nil
}
// VerifyContentHash checks the event against the content hash it carries.
//
// It returns ErrContentHashMismatch when the hash does not match, which a
// federation caller should answer by using the redacted copy of the event
// rather than by rejecting it.
func (e *Event) VerifyContentHash() error {
want, err := signing.DecodeBase64(e.fields.Hashes.SHA256)
if err != nil {
return fmt.Errorf("%w: hashes.sha256: %w", ErrInvalidEvent, err)
}
got, err := contentHash(e.raw)
if err != nil {
return err
}
if !bytes.Equal(want, got[:]) {
return ErrContentHashMismatch
}
return nil
}
// decodeObject parses an event into its properties, leaving every value as
// the bytes it arrived as so that nothing we do not understand is lost.
func decodeObject(raw []byte) (map[string]json.RawMessage, error) {
var obj map[string]json.RawMessage
if err := json.Unmarshal(raw, &obj); err != nil {
return nil, fmt.Errorf("%w: %w", ErrInvalidEvent, err)
}
if obj == nil {
return nil, fmt.Errorf("%w: event is null, not an object", ErrInvalidEvent)
}
return obj, nil
}
+222
View File
@@ -0,0 +1,222 @@
package event
import (
"encoding/json"
"fmt"
"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
)
// Redact returns the event with everything a redaction removes stripped
// off, leaving the properties the room version protects.
//
// The redacted event keeps the same event ID, because the ID is the
// reference hash of the redacted form in the first place — redacting an
// event does not rename it, and the signature on it still verifies. That is
// the whole point of the algorithm: a server can prove a redacted event is
// genuine without ever having seen its content.
//
// Redact says nothing about whether the redaction is *allowed*. That
// depends on the sender's power level or on their server having sent the
// original, and belongs to the roomserver along with the decision to apply
// it at all.
func (e *Event) Redact() (*Event, error) {
redacted, err := redactJSON(e.raw, e.rules)
if err != nil {
return nil, err
}
out := &Event{rules: e.rules, raw: redacted, id: e.id}
if err := json.Unmarshal(redacted, &out.fields); err != nil {
return nil, fmt.Errorf("%w: redacted event: %w", ErrInvalidEvent, err)
}
if e.rules.RedactsInContent && out.fields.Type == TypeRedaction {
var content struct {
Redacts string `json:"redacts"`
}
if err := json.Unmarshal(out.fields.Content, &content); err != nil {
return nil, fmt.Errorf("%w: redacted m.room.redaction content: %w", ErrInvalidEvent, err)
}
out.fields.Redacts = content.Redacts
}
return out, nil
}
// redactJSON applies the redaction algorithm to raw event JSON and returns
// canonical JSON.
func redactJSON(raw []byte, rules roomversion.Rules) ([]byte, error) {
obj, err := decodeObject(raw)
if err != nil {
return nil, err
}
var header struct {
Type string `json:"type"`
}
if err := json.Unmarshal(raw, &header); err != nil {
return nil, fmt.Errorf("%w: %w", ErrInvalidEvent, err)
}
keep := protectedTopLevelKeys(rules)
for k := range obj {
if !keep[k] {
delete(obj, k)
}
}
content, err := redactContent(header.Type, obj["content"], rules)
if err != nil {
return nil, err
}
obj["content"] = content
out, err := canonicaljson.Marshal(obj)
if err != nil {
return nil, fmt.Errorf("%w: redact: %w", ErrInvalidEvent, err)
}
return out, nil
}
// protectedTopLevelKeys returns the event properties a redaction keeps.
func protectedTopLevelKeys(rules roomversion.Rules) map[string]bool {
keep := map[string]bool{
"event_id": true,
"type": true,
"room_id": true,
"sender": true,
"state_key": true,
"content": true,
"hashes": true,
"signatures": true,
"depth": true,
"prev_events": true,
"auth_events": true,
"origin_server_ts": true,
}
if rules.RedactionProtectsLegacyKeys {
// Room versions up to 10 also protected these. None was
// load-bearing, and the top-level membership in particular let a
// redacted event keep asserting a membership its content no longer
// supported, so version 11 stopped protecting them.
keep["prev_state"] = true
keep["origin"] = true
keep["membership"] = true
}
return keep
}
// redactContent strips the content of an event down to the keys its type is
// allowed to keep.
func redactContent(eventType string, content json.RawMessage, rules roomversion.Rules) (json.RawMessage, error) {
empty := json.RawMessage("{}")
if len(content) == 0 {
return empty, nil
}
// m.room.create keeps all of its content from room version 11, so
// there is nothing to strip and nothing to re-encode.
if eventType == TypeCreate && rules.RedactionKeepsWholeCreateContent {
return content, nil
}
keep := protectedContentKeys(eventType, rules)
keepsThirdPartyInvite := eventType == TypeMember && rules.RedactionKeepsThirdPartyInviteSigned
if len(keep) == 0 && !keepsThirdPartyInvite {
return empty, nil
}
var obj map[string]json.RawMessage
if err := json.Unmarshal(content, &obj); err != nil {
return nil, fmt.Errorf("%w: content: %w", ErrInvalidEvent, err)
}
for k := range obj {
if !keep[k] {
delete(obj, k)
}
}
if keepsThirdPartyInvite {
if signed, ok := thirdPartyInviteSigned(content); ok {
obj["third_party_invite"] = signed
}
}
out, err := json.Marshal(obj)
if err != nil {
return nil, fmt.Errorf("%w: redact content: %w", ErrInvalidEvent, err)
}
return out, nil
}
// protectedContentKeys returns the content keys a redaction keeps for an
// event type. An unlisted type keeps nothing.
func protectedContentKeys(eventType string, rules roomversion.Rules) map[string]bool {
keep := map[string]bool{}
switch eventType {
case TypeMember:
keep["membership"] = true
if rules.RedactionKeepsJoinAuthorisedViaUsersServer {
// Without this, a redacted join in a restricted room could
// never be re-authorised, because the proof that somebody with
// the power to invite had allowed it was in the key that went
// away.
keep["join_authorised_via_users_server"] = true
}
case TypeCreate:
// Before room version 11 only the creator survived, which threw
// away room_version — the one property a server needs in order to
// read the rest of the room.
keep["creator"] = true
case TypeJoinRules:
keep["join_rule"] = true
if rules.RedactionKeepsJoinRulesAllow {
keep["allow"] = true
}
case TypePowerLevels:
for _, k := range []string{
"ban", "events", "events_default", "kick",
"redact", "state_default", "users", "users_default",
} {
keep[k] = true
}
if rules.RedactionKeepsPowerLevelsInvite {
keep["invite"] = true
}
case TypeHistoryVisibility:
keep["history_visibility"] = true
case TypeRedaction:
if rules.RedactionKeepsRedacts {
keep["redacts"] = true
}
case TypeAliases:
if rules.RedactionKeepsAliases {
keep["aliases"] = true
}
}
return keep
}
// thirdPartyInviteSigned returns {"signed": ...} from an m.room.member
// event's third_party_invite, if it has one.
//
// From room version 11 this much of a third-party invite survives
// redaction: it is the signature chain that proves the invite was issued by
// an identity server, and an event that loses it can never be authorised
// again.
func thirdPartyInviteSigned(content json.RawMessage) (json.RawMessage, bool) {
var parsed struct {
ThirdPartyInvite map[string]json.RawMessage `json:"third_party_invite"`
}
if err := json.Unmarshal(content, &parsed); err != nil {
return nil, false
}
signed, ok := parsed.ThirdPartyInvite["signed"]
if !ok {
return nil, false
}
out, err := json.Marshal(map[string]json.RawMessage{"signed": signed})
if err != nil {
return nil, false
}
return out, true
}
+113
View File
@@ -0,0 +1,113 @@
package event
import (
"crypto/ed25519"
"encoding/base64"
"encoding/json"
"fmt"
"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
)
// hashAndSign adds a content hash and a signature to a bare event.
//
// The order matters and is not the obvious one. The content hash is added
// first, so that it covers every property of the unredacted event. The
// event is then *redacted* and that copy is signed, so the signature covers
// only the properties a redaction keeps — plus the hashes, which cover the
// rest. The signature is then copied back onto the full event.
//
// The result is an event whose signature still verifies after someone
// redacts it, which is what lets a server hand out a redacted event and
// have the recipient believe it.
func hashAndSign(raw []byte, rules roomversion.Rules, serverName string, key *signing.KeyPair) ([]byte, error) {
obj, err := decodeObject(raw)
if err != nil {
return nil, err
}
sum, err := contentHash(raw)
if err != nil {
return nil, err
}
obj["hashes"], err = json.Marshal(hashes{SHA256: base64.RawStdEncoding.EncodeToString(sum[:])})
if err != nil {
return nil, fmt.Errorf("%w: encode hashes: %w", ErrInvalidEvent, err)
}
hashed, err := canonicaljson.Marshal(obj)
if err != nil {
return nil, fmt.Errorf("%w: %w", ErrInvalidEvent, err)
}
redacted, err := redactJSON(hashed, rules)
if err != nil {
return nil, err
}
signed, err := signing.SignJSON(serverName, key, redacted)
if err != nil {
return nil, err
}
sigs, err := signing.ExtractSignatures(signed)
if err != nil {
return nil, err
}
if obj["signatures"], err = json.Marshal(sigs); err != nil {
return nil, fmt.Errorf("%w: encode signatures: %w", ErrInvalidEvent, err)
}
return canonicaljson.Marshal(obj)
}
// VerifySignature checks the event for a valid signature from serverName
// made with keyID.
//
// The event is redacted first, because that is the form that was signed.
// This holds whether we were given the full event or a redacted copy, which
// is why a redacted event can still be shown to be genuine.
func (e *Event) VerifySignature(serverName, keyID string, pub ed25519.PublicKey) error {
redacted, err := redactJSON(e.raw, e.rules)
if err != nil {
return err
}
return signing.VerifyJSON(serverName, keyID, pub, redacted)
}
// Sign returns the event with an additional signature from serverName.
//
// A normal event is signed once, by the sender's own server, when it is
// built. A second signature is needed in the few places where another
// server has to vouch for an event it did not send: the invited server
// countersigning an invite, and a third-party invite exchanged through a
// server other than the sender's.
func (e *Event) Sign(serverName string, key *signing.KeyPair) (*Event, error) {
redacted, err := redactJSON(e.raw, e.rules)
if err != nil {
return nil, err
}
signed, err := signing.SignJSON(serverName, key, redacted)
if err != nil {
return nil, err
}
sigs, err := signing.ExtractSignatures(signed)
if err != nil {
return nil, err
}
obj, err := decodeObject(e.raw)
if err != nil {
return nil, err
}
if obj["signatures"], err = json.Marshal(sigs); err != nil {
return nil, fmt.Errorf("%w: encode signatures: %w", ErrInvalidEvent, err)
}
out, err := canonicaljson.Marshal(obj)
if err != nil {
return nil, fmt.Errorf("%w: %w", ErrInvalidEvent, err)
}
// Re-parse rather than patching the Event in place: an Event is
// immutable, and the signature block is one of the things Parse
// validates.
return Parse(out, e.rules)
}
+148
View File
@@ -0,0 +1,148 @@
package event
import (
"encoding/json"
"fmt"
"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
"github.com/dariuszjaniak/warp/server/internal/matrix/id"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
)
// The limits the spec puts on an event.
const (
// MaxEventSize is the largest an event may be as canonical JSON,
// signatures included. 64 KiB is generous for a message and small
// enough that a room's state cannot be used to exhaust a server.
MaxEventSize = 65536
// MaxFieldLength caps the type and state_key properties.
MaxFieldLength = 255
// MaxPrevEvents and MaxAuthEvents cap the DAG references an event may
// carry. The auth events selection algorithm never needs more than a
// handful, so a longer list is a sign of something wrong rather than a
// legitimate event.
MaxPrevEvents = 20
MaxAuthEvents = 10
// MaxDepth is the largest depth an event may claim. Once a room gets
// here every later event stays at the limit.
MaxDepth = canonicaljson.MaxInt
)
// validate applies the structural rules: the properties the spec marks
// required are present and of the right type, the identifiers are
// well-formed, and nothing is over its size limit.
//
// It deliberately does not look at the room state. Everything checkable
// from the event alone is checked here so that the authorisation rules can
// assume a well-formed event and concern themselves only with permission.
func (e *Event) validate() error {
if e.rules.EventIDFormat == roomversion.EventIDFormatExplicit {
// Unreachable today: Get refuses to return rules for room versions
// 1 and 2. The check is here so that adding them to the registry
// fails loudly rather than computing nonsense event IDs.
return fmt.Errorf("%w: room version %s carries event IDs in the event, which Warp cannot read yet",
ErrInvalidEvent, e.rules.Version)
}
if n := len(e.raw); n > MaxEventSize {
return fmt.Errorf("%w: event is %d bytes, the limit is %d", ErrInvalidEvent, n, MaxEventSize)
}
if e.fields.Type == "" {
return fmt.Errorf("%w: type is required", ErrInvalidEvent)
}
if n := len(e.fields.Type); n > MaxFieldLength {
return fmt.Errorf("%w: type is %d bytes, the limit is %d", ErrInvalidEvent, n, MaxFieldLength)
}
if err := id.ValidateUserID(e.fields.Sender); err != nil {
return fmt.Errorf("%w: sender: %w", ErrInvalidEvent, err)
}
if sk, ok := e.StateKey(); ok {
if n := len(sk); n > MaxFieldLength {
return fmt.Errorf("%w: state_key is %d bytes, the limit is %d", ErrInvalidEvent, n, MaxFieldLength)
}
}
if err := e.validateRoomID(); err != nil {
return err
}
if err := e.validateContent(); err != nil {
return err
}
if err := e.validateDAGReferences(); err != nil {
return err
}
if e.fields.Depth > MaxDepth {
return fmt.Errorf("%w: depth %d is above the limit of %d", ErrInvalidEvent, e.fields.Depth, MaxDepth)
}
if e.fields.Hashes.SHA256 == "" {
return fmt.Errorf("%w: hashes.sha256 is required", ErrInvalidEvent)
}
if len(e.fields.Signatures) == 0 {
return fmt.Errorf("%w: signatures is required", ErrInvalidEvent)
}
return nil
}
// validateRoomID checks the room_id property, which from room version 12 is
// absent on the create event because the room ID is derived from it.
func (e *Event) validateRoomID() error {
isCreate := e.fields.Type == TypeCreate
if e.rules.RoomIDIsCreateEventID && isCreate {
if e.fields.RoomID != "" {
// Rule 1.2 for this room version rejects it as well; failing
// here too means we never compute an ID for an event whose
// room is ambiguous.
return fmt.Errorf("%w: an m.room.create event in a version %s room must not have a room_id",
ErrInvalidEvent, e.rules.Version)
}
return nil
}
if err := id.ValidateRoomID(e.fields.RoomID); err != nil {
return fmt.Errorf("%w: room_id: %w", ErrInvalidEvent, err)
}
return nil
}
// validateContent checks that content is present and is an object. The
// spec marks it required on every event, and the authorisation rules and
// the redaction algorithm both read into it.
func (e *Event) validateContent() error {
if len(e.fields.Content) == 0 {
return fmt.Errorf("%w: content is required", ErrInvalidEvent)
}
var obj map[string]json.RawMessage
if err := json.Unmarshal(e.fields.Content, &obj); err != nil {
return fmt.Errorf("%w: content must be an object: %w", ErrInvalidEvent, err)
}
if obj == nil {
return fmt.Errorf("%w: content must be an object, not null", ErrInvalidEvent)
}
return nil
}
// validateDAGReferences checks prev_events and auth_events: within their
// limits, and every entry a well-formed event ID.
func (e *Event) validateDAGReferences() error {
if n := len(e.fields.PrevEvents); n > MaxPrevEvents {
return fmt.Errorf("%w: %d prev_events, the limit is %d", ErrInvalidEvent, n, MaxPrevEvents)
}
if n := len(e.fields.AuthEvents); n > MaxAuthEvents {
return fmt.Errorf("%w: %d auth_events, the limit is %d", ErrInvalidEvent, n, MaxAuthEvents)
}
for _, eventID := range e.fields.PrevEvents {
if err := id.ValidateEventID(eventID); err != nil {
return fmt.Errorf("%w: prev_events: %w", ErrInvalidEvent, err)
}
}
for _, eventID := range e.fields.AuthEvents {
if err := id.ValidateEventID(eventID); err != nil {
return fmt.Errorf("%w: auth_events: %w", ErrInvalidEvent, err)
}
}
return nil
}
+139
View File
@@ -0,0 +1,139 @@
// Package id validates the Matrix identifiers that appear inside events.
//
// The checks here are deliberately laxer than the ones in userapi, which
// guard account creation on this server and enforce the current, strict
// localpart grammar. Events are different: a room can contain user IDs
// minted years ago by servers that allowed upper case and other characters
// the grammar has since dropped, and the spec requires receivers to accept
// them. Rejecting them would mean refusing to authorise perfectly valid
// history, so the rules below are the "historical" ones — a sigil, a
// non-empty localpart, a colon, a server name, and a length limit.
package id
import (
"errors"
"fmt"
"strings"
)
// MaxLength is the spec's cap on a user ID, room ID, alias or event ID,
// sigil and server name included.
const MaxLength = 255
// ErrInvalidID is the root of every error this package returns.
var ErrInvalidID = errors.New("invalid Matrix identifier")
// ValidateUserID checks a user ID against the historical grammar:
// "@localpart:server_name", at most 255 bytes, with no NUL anywhere.
//
// It is the check behind the authorisation rules that call for a "valid
// user ID", such as the keys of m.room.power_levels users.
func ValidateUserID(s string) error {
_, _, err := SplitUserID(s)
return err
}
// SplitUserID splits a user ID into its localpart and server name, without
// the sigil or the colon.
func SplitUserID(s string) (localpart, serverName string, err error) {
if err := checkCommon("user ID", s, '@'); err != nil {
return "", "", err
}
localpart, serverName, ok := strings.Cut(s[1:], ":")
if !ok {
return "", "", fmt.Errorf("%w: user ID %q has no server name; expected @localpart:server", ErrInvalidID, s)
}
if localpart == "" {
return "", "", fmt.Errorf("%w: user ID %q has an empty localpart", ErrInvalidID, s)
}
if serverName == "" {
return "", "", fmt.Errorf("%w: user ID %q has an empty server name", ErrInvalidID, s)
}
return localpart, serverName, nil
}
// ServerNameOf returns the server name from a user ID.
func ServerNameOf(userID string) (string, error) {
_, serverName, err := SplitUserID(userID)
return serverName, err
}
// SameServer reports whether two user IDs belong to the same homeserver.
//
// Server names are compared byte for byte: the spec says they are
// case-sensitive, and treating "Example.org" and "example.org" as one
// server here would quietly widen rules that turn on a domain match, such
// as the m.federate check and the right to redact another user's event.
func SameServer(userIDA, userIDB string) bool {
a, errA := ServerNameOf(userIDA)
b, errB := ServerNameOf(userIDB)
if errA != nil || errB != nil {
return false
}
return a == b
}
// ValidateRoomID checks a room ID: a '!' sigil and at most 255 bytes.
//
// Whether the rest of it is "opaque:server" or the hash of the create
// event depends on the room version, and in the hashed form there is
// nothing left to check beyond the sigil and the length — so the
// version-specific part of the check lives with the authorisation rules,
// which have the create event to compare against.
func ValidateRoomID(s string) error {
if err := checkCommon("room ID", s, '!'); err != nil {
return err
}
if len(s) == 1 {
return fmt.Errorf("%w: room ID %q is only a sigil", ErrInvalidID, s)
}
return nil
}
// ServerNameOfRoomID returns the server name from a room ID of the
// "!opaque:server" form used up to room version 11.
//
// It fails for a room version 12 room ID, which is a hash of the create
// event and names no server at all — the caller should only ask in the
// versions where the answer exists.
func ServerNameOfRoomID(roomID string) (string, error) {
if err := ValidateRoomID(roomID); err != nil {
return "", err
}
_, serverName, ok := strings.Cut(roomID[1:], ":")
if !ok || serverName == "" {
return "", fmt.Errorf("%w: room ID %q names no server", ErrInvalidID, roomID)
}
return serverName, nil
}
// ValidateEventID checks an event ID: a '$' sigil and at most 255 bytes.
func ValidateEventID(s string) error {
if err := checkCommon("event ID", s, '$'); err != nil {
return err
}
if len(s) == 1 {
return fmt.Errorf("%w: event ID %q is only a sigil", ErrInvalidID, s)
}
return nil
}
// checkCommon applies the three rules every identifier shares.
func checkCommon(kind, s string, sigil byte) error {
if s == "" {
return fmt.Errorf("%w: %s must not be empty", ErrInvalidID, kind)
}
if s[0] != sigil {
return fmt.Errorf("%w: %s %q does not start with %q", ErrInvalidID, kind, s, string(sigil))
}
if len(s) > MaxLength {
return fmt.Errorf("%w: %s is %d bytes, the limit is %d", ErrInvalidID, kind, len(s), MaxLength)
}
// A NUL would terminate the string in most of the software that ever
// sees it, which makes two identifiers that differ after it look the
// same to some servers and not to others.
if strings.IndexByte(s, 0) >= 0 {
return fmt.Errorf("%w: %s %q contains a NUL byte", ErrInvalidID, kind, s)
}
return nil
}
+184
View File
@@ -0,0 +1,184 @@
package id
import (
"errors"
"strings"
"testing"
)
func TestSplitUserID(t *testing.T) {
cases := []struct {
in string
localpart string
serverName string
}{
{"@alice:warp.test", "alice", "warp.test"},
{"@alice:warp.test:8448", "alice", "warp.test:8448"},
// Historical IDs: characters the current grammar forbids still have
// to be accepted, because rooms are full of them.
{"@Alice:warp.test", "Alice", "warp.test"},
{"@a b:warp.test", "a b", "warp.test"},
{"@😀:warp.test", "😀", "warp.test"},
}
for _, tc := range cases {
localpart, serverName, err := SplitUserID(tc.in)
if err != nil {
t.Errorf("SplitUserID(%q) = %v", tc.in, err)
continue
}
if localpart != tc.localpart || serverName != tc.serverName {
t.Errorf("SplitUserID(%q) = %q, %q; want %q, %q",
tc.in, localpart, serverName, tc.localpart, tc.serverName)
}
}
}
func TestValidateUserIDRejects(t *testing.T) {
cases := map[string]string{
"": "must not be empty",
"alice:warp.test": "does not start with",
"#alice:warp.test": "does not start with",
"@alice": "no server name",
"@:warp.test": "empty localpart",
"@alice:": "empty server name",
"@alice\x00:warp.test": "NUL",
"@" + strings.Repeat("a", 255): "the limit is",
}
for in, want := range cases {
err := ValidateUserID(in)
if err == nil {
t.Errorf("ValidateUserID(%q) was accepted, want an error", in)
continue
}
if !strings.Contains(err.Error(), want) {
t.Errorf("ValidateUserID(%q) = %q, want it to mention %q", in, err, want)
}
if !errors.Is(err, ErrInvalidID) {
t.Errorf("ValidateUserID(%q) does not wrap ErrInvalidID: %v", in, err)
}
}
}
func TestUserIDLengthLimit(t *testing.T) {
// Exactly at the limit is allowed; one byte over is not.
atLimit := "@" + strings.Repeat("a", MaxLength-len("@:warp.test")) + ":warp.test"
if len(atLimit) != MaxLength {
t.Fatalf("the test built a %d-byte ID, want %d", len(atLimit), MaxLength)
}
if err := ValidateUserID(atLimit); err != nil {
t.Errorf("a %d-byte user ID was rejected: %v", MaxLength, err)
}
if err := ValidateUserID(atLimit + "a"); err == nil {
t.Error("a user ID one byte over the limit was accepted")
}
}
func TestServerNameOf(t *testing.T) {
got, err := ServerNameOf("@alice:warp.test")
if err != nil {
t.Fatal(err)
}
if got != "warp.test" {
t.Errorf("ServerNameOf = %q, want warp.test", got)
}
if _, err := ServerNameOf("nonsense"); err == nil {
t.Error("ServerNameOf accepted a string that is not a user ID")
}
}
func TestSameServer(t *testing.T) {
cases := []struct {
a, b string
want bool
}{
{"@alice:warp.test", "@bob:warp.test", true},
{"@alice:warp.test", "@alice:warp.test", true},
{"@alice:warp.test", "@bob:other.test", false},
// Server names are case-sensitive by spec, and the rules that turn
// on a domain match must not be widened by a capital letter.
{"@alice:warp.test", "@bob:Warp.test", false},
{"@alice:warp.test", "not-a-user-id", false},
{"", "", false},
}
for _, tc := range cases {
if got := SameServer(tc.a, tc.b); got != tc.want {
t.Errorf("SameServer(%q, %q) = %v, want %v", tc.a, tc.b, got, tc.want)
}
}
}
func TestValidateRoomID(t *testing.T) {
valid := []string{
"!abc:warp.test",
// From room version 12 a room ID is a hash with no domain at all.
"!Nhcu5BS-UMnFX7hBVfVSoXiD7OgH6iRT-xyIuqDnpYQ",
}
for _, s := range valid {
if err := ValidateRoomID(s); err != nil {
t.Errorf("ValidateRoomID(%q) = %v", s, err)
}
}
invalid := map[string]string{
"": "must not be empty",
"abc:warp.test": "does not start with",
"@abc:warp.test": "does not start with",
"!": "only a sigil",
"!abc\x00:warp.test": "NUL",
"!" + strings.Repeat("a", 255): "the limit is",
}
for s, want := range invalid {
err := ValidateRoomID(s)
if err == nil {
t.Errorf("ValidateRoomID(%q) was accepted, want an error", s)
continue
}
if !strings.Contains(err.Error(), want) {
t.Errorf("ValidateRoomID(%q) = %q, want it to mention %q", s, err, want)
}
}
}
func TestServerNameOfRoomID(t *testing.T) {
got, err := ServerNameOfRoomID("!abc:warp.test")
if err != nil {
t.Fatal(err)
}
if got != "warp.test" {
t.Errorf("ServerNameOfRoomID = %q, want warp.test", got)
}
// A version 12 room ID names no server, and saying so is the point:
// the caller must only ask in the versions where there is an answer.
if _, err := ServerNameOfRoomID("!Nhcu5BS-UMnFX7hBVfVSoXiD7OgH6iRT"); err == nil {
t.Error("ServerNameOfRoomID found a server in a room ID that has none")
}
}
func TestValidateEventID(t *testing.T) {
valid := []string{"$Rqnc-F-dvnEYJTyHq_iKxU2bZ1CI92-kuZq3a5lr5Zg", "$abc:warp.test"}
for _, s := range valid {
if err := ValidateEventID(s); err != nil {
t.Errorf("ValidateEventID(%q) = %v", s, err)
}
}
invalid := map[string]string{
"": "must not be empty",
"abc": "does not start with",
"!abc": "does not start with",
"$": "only a sigil",
"$abc\x00": "NUL",
"$" + strings.Repeat("a", 255): "the limit is",
}
for s, want := range invalid {
err := ValidateEventID(s)
if err == nil {
t.Errorf("ValidateEventID(%q) was accepted, want an error", s)
continue
}
if !strings.Contains(err.Error(), want) {
t.Errorf("ValidateEventID(%q) = %q, want it to mention %q", s, err, want)
}
}
}
@@ -0,0 +1,510 @@
// Package roomversion is the registry of Matrix room versions and of what
// each one changes.
//
// A room version fixes the rules every server in that room must follow:
// how event IDs are derived, which keys survive a redaction, and which
// authorisation rules apply. Rooms never change version in place, so a
// server that wants to take part in old rooms has to keep every old rule
// set working — the differences are small but they are not optional, and a
// server that gets one wrong corrupts the room's history for everybody.
//
// Rather than branch on the version string in a dozen places, each version
// is described once here as a Rules value, and the event, auth and stateres
// packages ask it what to do.
//
// # Supported versions
//
// Versions 3 to 12 are implemented. Versions 1 and 2 are listed as known
// but unsupported: they carry the event ID inside the event, write
// prev_events and auth_events as [id, hashes] tuples rather than plain
// lists, and use state resolution v1, which is enough of a separate event
// format to be its own piece of work. It is only needed to join rooms
// created before 2018 on other servers, so it waits for federation
// (PLAN.md phase 5) and the registry rejects those rooms clearly until then.
package roomversion
import (
"errors"
"fmt"
"slices"
"strings"
)
// Version is a room version identifier, such as "11". It is an opaque
// string by spec: do not compare it with < or parse it as a number.
type Version string
// The versions this package knows about.
const (
V1 Version = "1"
V2 Version = "2"
V3 Version = "3"
V4 Version = "4"
V5 Version = "5"
V6 Version = "6"
V7 Version = "7"
V8 Version = "8"
V9 Version = "9"
V10 Version = "10"
V11 Version = "11"
V12 Version = "12"
)
// Default is the version `createRoom` uses when the client does not ask for
// one.
//
// The spec says servers SHOULD default to 12, but also asks them to stay on
// 11 for a while after the 1.16 release so that clients and bridges catch
// up (MSC4304). We follow that advice: 12 is implemented and a client can
// ask for it, but a plain createRoom still makes an 11 room.
const Default = V11
// Errors from Get. They are distinct because the answers differ: an
// unknown version is a client typo or a room from the future, while an
// unsupported one is a gap in this server.
var (
// ErrUnknownVersion means no such room version exists, as far as this
// build of Warp knows.
ErrUnknownVersion = errors.New("unknown room version")
// ErrUnsupportedVersion means the version exists but Warp cannot take
// part in rooms that use it.
ErrUnsupportedVersion = errors.New("unsupported room version")
)
// EventIDFormat says how the event ID of an event is arrived at.
type EventIDFormat int
const (
// EventIDFormatExplicit carries the ID in the event's event_id
// property, as "$random:server" (room versions 1 and 2).
EventIDFormatExplicit EventIDFormat = iota
// EventIDFormatSHA256 computes the ID from the reference hash, in
// unpadded standard base64 (room version 3).
EventIDFormatSHA256
// EventIDFormatSHA256URLSafe computes the ID from the reference hash,
// in URL-safe unpadded base64 (room version 4 and later). The change
// was made because an ID containing '/' cannot be put in a path.
EventIDFormatSHA256URLSafe
)
// StateResVersion says which state resolution algorithm a room uses.
type StateResVersion int
const (
// StateResV1 is the original algorithm, used only by room version 1.
StateResV1 StateResVersion = iota
// StateResV2 is the algorithm from room version 2 onwards.
StateResV2
// StateResV2Subgraph is state resolution v2 as amended by room version
// 12: iterative auth checks start from an empty state map, and the full
// conflicted set also contains the conflicted state subgraph. Both
// changes exist to make state resets rarer.
StateResV2Subgraph
)
// Rules is everything that varies between room versions. A zero Rules is
// not usable; get one from Get or MustGet.
type Rules struct {
// Version is the identifier these rules belong to.
Version Version
// Supported reports whether Warp can take part in rooms of this
// version. Get refuses to return rules that are not supported, so a
// Rules in hand is always supported; the field exists so that Known
// can report the full picture.
Supported bool
// EventIDFormat says where an event's ID comes from.
EventIDFormat EventIDFormat
// StateRes says which state resolution algorithm applies.
StateRes StateResVersion
// StrictCanonicalJSON makes a malformed number or a non-UTF-8 string a
// hard rejection rather than something to tolerate (version 6+). In
// older rooms such events exist and still have to be processed.
StrictCanonicalJSON bool
// EnforceSigningKeyValidity requires a signing key's valid_until_ts to
// be at least the event's origin_server_ts (version 5+).
EnforceSigningKeyValidity bool
// --- Authorisation ---
// AliasesEventAuth gives m.room.aliases its own authorisation rule,
// under which any server could edit its own alias list (versions 1 to
// 5). Version 6 dropped it, and with it the ability to put another
// server's aliases in a room.
AliasesEventAuth bool
// CreateEventRequiresCreator rejects an m.room.create event whose
// content has no creator property (versions 1 to 10). Version 11
// removed the property, since it only ever repeated sender.
CreateEventRequiresCreator bool
// KnockJoinRule enables the knock membership and join rule (version 7+).
KnockJoinRule bool
// RestrictedJoinRule enables the restricted join rule, where
// membership of another room grants entry, authorised by a server that
// can see that membership (version 8+).
RestrictedJoinRule bool
// KnockRestrictedJoinRule enables knock_restricted, the two combined
// (version 10+).
KnockRestrictedJoinRule bool
// RequireIntegerPowerLevels rejects a power level given as a string
// (version 10+). Earlier versions accepted "50" and parsed it, which
// meant two servers could disagree about what a value meant.
RequireIntegerPowerLevels bool
// RoomIDIsCreateEventID makes the room ID the create event's own ID
// with a '!' sigil instead of '$', so it carries no domain. The create
// event has no room_id of its own (version 12+).
RoomIDIsCreateEventID bool
// CreateEventExcludedFromAuthEvents keeps m.room.create out of every
// event's auth_events, because the room ID now identifies it (version
// 12+).
CreateEventExcludedFromAuthEvents bool
// CreatorsHaveInfinitePower gives the room's creators a power level
// above any that can be written down, and forbids naming them in
// m.room.power_levels (version 12+).
CreatorsHaveInfinitePower bool
// AdditionalCreators allows m.room.create content to name further
// creators alongside the sender (version 12+).
AdditionalCreators bool
// --- Redaction ---
// RedactionProtectsLegacyKeys keeps the top-level prev_state, origin
// and membership properties through a redaction (versions 1 to 10).
// Version 11 stopped protecting them: none was load-bearing, and
// membership in particular let a redaction preserve a claim the
// content no longer backed up.
RedactionProtectsLegacyKeys bool
// RedactionKeepsWholeCreateContent keeps all of an m.room.create
// event's content (version 11+). Before that only creator survived,
// which threw away the room version — the one thing a server needs to
// read the rest of the room.
RedactionKeepsWholeCreateContent bool
// RedactsInContent puts the target of an m.room.redaction event in
// content.redacts rather than at the top level (version 11+). The
// property moved so that the redaction algorithm, and therefore the
// event's hashes and signatures, would cover it at all.
RedactsInContent bool
// RedactionKeepsRedacts keeps content.redacts on an m.room.redaction
// event (version 11+), where the property moved so that it would be
// covered by hashes and signatures at all.
RedactionKeepsRedacts bool
// RedactionKeepsPowerLevelsInvite keeps content.invite on an
// m.room.power_levels event (version 11+).
RedactionKeepsPowerLevelsInvite bool
// RedactionKeepsThirdPartyInviteSigned keeps
// content.third_party_invite.signed on an m.room.member event
// (version 11+), which is what proves a third-party invite was real.
RedactionKeepsThirdPartyInviteSigned bool
// RedactionKeepsJoinAuthorisedViaUsersServer keeps
// content.join_authorised_via_users_server on an m.room.member event
// (version 9+). Version 8 redacted it, which made a redacted join in a
// restricted room fail to authorise for ever after.
RedactionKeepsJoinAuthorisedViaUsersServer bool
// RedactionKeepsJoinRulesAllow keeps content.allow on an
// m.room.join_rules event (version 8+).
RedactionKeepsJoinRulesAllow bool
// RedactionKeepsAliases keeps content.aliases on an m.room.aliases
// event (versions 1 to 5).
RedactionKeepsAliases bool
}
// String returns the version identifier.
func (r Rules) String() string { return string(r.Version) }
// versions is the registry, in the order Known reports them.
//
// Each entry is written out in full rather than derived from the one before
// it. It is more lines, but a reader can see what a single version does
// without replaying the whole history, and a new version cannot change an
// old one by accident.
var versions = []Rules{
{
Version: V1,
Supported: false,
EventIDFormat: EventIDFormatExplicit,
StateRes: StateResV1,
AliasesEventAuth: true,
CreateEventRequiresCreator: true,
RedactionProtectsLegacyKeys: true,
RedactionKeepsAliases: true,
},
{
Version: V2,
Supported: false,
EventIDFormat: EventIDFormatExplicit,
StateRes: StateResV2,
AliasesEventAuth: true,
CreateEventRequiresCreator: true,
RedactionProtectsLegacyKeys: true,
RedactionKeepsAliases: true,
},
{
Version: V3,
Supported: true,
EventIDFormat: EventIDFormatSHA256,
StateRes: StateResV2,
AliasesEventAuth: true,
CreateEventRequiresCreator: true,
RedactionProtectsLegacyKeys: true,
RedactionKeepsAliases: true,
},
{
Version: V4,
Supported: true,
EventIDFormat: EventIDFormatSHA256URLSafe,
StateRes: StateResV2,
AliasesEventAuth: true,
CreateEventRequiresCreator: true,
RedactionProtectsLegacyKeys: true,
RedactionKeepsAliases: true,
},
{
Version: V5,
Supported: true,
EventIDFormat: EventIDFormatSHA256URLSafe,
StateRes: StateResV2,
EnforceSigningKeyValidity: true,
AliasesEventAuth: true,
CreateEventRequiresCreator: true,
RedactionProtectsLegacyKeys: true,
RedactionKeepsAliases: true,
},
{
Version: V6,
Supported: true,
EventIDFormat: EventIDFormatSHA256URLSafe,
StateRes: StateResV2,
StrictCanonicalJSON: true,
EnforceSigningKeyValidity: true,
CreateEventRequiresCreator: true,
RedactionProtectsLegacyKeys: true,
},
{
Version: V7,
Supported: true,
EventIDFormat: EventIDFormatSHA256URLSafe,
StateRes: StateResV2,
StrictCanonicalJSON: true,
EnforceSigningKeyValidity: true,
CreateEventRequiresCreator: true,
KnockJoinRule: true,
RedactionProtectsLegacyKeys: true,
},
{
Version: V8,
Supported: true,
EventIDFormat: EventIDFormatSHA256URLSafe,
StateRes: StateResV2,
StrictCanonicalJSON: true,
EnforceSigningKeyValidity: true,
CreateEventRequiresCreator: true,
KnockJoinRule: true,
RestrictedJoinRule: true,
RedactionProtectsLegacyKeys: true,
RedactionKeepsJoinRulesAllow: true,
},
{
Version: V9,
Supported: true,
EventIDFormat: EventIDFormatSHA256URLSafe,
StateRes: StateResV2,
StrictCanonicalJSON: true,
EnforceSigningKeyValidity: true,
CreateEventRequiresCreator: true,
KnockJoinRule: true,
RestrictedJoinRule: true,
RedactionProtectsLegacyKeys: true,
RedactionKeepsJoinRulesAllow: true,
RedactionKeepsJoinAuthorisedViaUsersServer: true,
},
{
Version: V10,
Supported: true,
EventIDFormat: EventIDFormatSHA256URLSafe,
StateRes: StateResV2,
StrictCanonicalJSON: true,
EnforceSigningKeyValidity: true,
CreateEventRequiresCreator: true,
KnockJoinRule: true,
RestrictedJoinRule: true,
KnockRestrictedJoinRule: true,
RequireIntegerPowerLevels: true,
RedactionProtectsLegacyKeys: true,
RedactionKeepsJoinRulesAllow: true,
RedactionKeepsJoinAuthorisedViaUsersServer: true,
},
{
Version: V11,
Supported: true,
EventIDFormat: EventIDFormatSHA256URLSafe,
StateRes: StateResV2,
StrictCanonicalJSON: true,
EnforceSigningKeyValidity: true,
KnockJoinRule: true,
RestrictedJoinRule: true,
KnockRestrictedJoinRule: true,
RequireIntegerPowerLevels: true,
RedactsInContent: true,
RedactionKeepsWholeCreateContent: true,
RedactionKeepsRedacts: true,
RedactionKeepsPowerLevelsInvite: true,
RedactionKeepsThirdPartyInviteSigned: true,
RedactionKeepsJoinRulesAllow: true,
RedactionKeepsJoinAuthorisedViaUsersServer: true,
},
{
Version: V12,
Supported: true,
EventIDFormat: EventIDFormatSHA256URLSafe,
StateRes: StateResV2Subgraph,
StrictCanonicalJSON: true,
EnforceSigningKeyValidity: true,
KnockJoinRule: true,
RestrictedJoinRule: true,
KnockRestrictedJoinRule: true,
RequireIntegerPowerLevels: true,
RedactsInContent: true,
RoomIDIsCreateEventID: true,
CreateEventExcludedFromAuthEvents: true,
CreatorsHaveInfinitePower: true,
AdditionalCreators: true,
RedactionKeepsWholeCreateContent: true,
RedactionKeepsRedacts: true,
RedactionKeepsPowerLevelsInvite: true,
RedactionKeepsThirdPartyInviteSigned: true,
RedactionKeepsJoinRulesAllow: true,
RedactionKeepsJoinAuthorisedViaUsersServer: true,
},
}
// index is versions keyed by identifier, built once at startup.
var index = func() map[Version]Rules {
m := make(map[Version]Rules, len(versions))
for _, v := range versions {
m[v.Version] = v
}
return m
}()
// Get returns the rules for a room version.
//
// It fails for a version Warp does not support, so every Rules in
// circulation is one we can actually act on.
func Get(v Version) (Rules, error) {
rules, ok := index[v]
if !ok {
return Rules{}, fmt.Errorf("%w: %q", ErrUnknownVersion, v)
}
if !rules.Supported {
return Rules{}, fmt.Errorf("%w: %q", ErrUnsupportedVersion, v)
}
return rules, nil
}
// MustGet is Get for a version fixed in the source, such as a test or
// Default. It panics rather than returning an error, because a typo in a
// constant is a bug to fix, not a condition to handle.
func MustGet(v Version) Rules {
rules, err := Get(v)
if err != nil {
panic(err)
}
return rules
}
// IsSupported reports whether Warp can take part in rooms of this version.
// It is the test behind the authorisation rule that rejects an
// m.room.create event naming a version we do not recognise.
func IsSupported(v Version) bool {
return index[v].Supported
}
// Supported lists the versions Warp can take part in, oldest first. It
// feeds the room_versions capability.
func Supported() []Version {
out := make([]Version, 0, len(versions))
for _, v := range versions {
if v.Supported {
out = append(out, v.Version)
}
}
return out
}
// Known lists every version in the registry, supported or not.
func Known() []Rules {
return slices.Clone(versions)
}
// MaxVersionLength is the spec's cap on a room version identifier.
const MaxVersionLength = 32
// Validate checks a string against the room version grammar: 1 to 32
// characters of lowercase letters, digits, '.' and '-'.
//
// It is for strings arriving from a client or another server, where the
// answer "that is not a room version at all" is more useful than "unknown
// room version". Note that a valid identifier need not be one we know.
func Validate(s string) error {
if s == "" {
return errors.New("room version must not be empty")
}
if n := len([]rune(s)); n > MaxVersionLength {
return fmt.Errorf("room version is %d characters, the limit is %d", n, MaxVersionLength)
}
for _, r := range s {
switch {
case r >= 'a' && r <= 'z', r >= '0' && r <= '9', r == '.', r == '-':
default:
return fmt.Errorf("room version %q contains %q; only lowercase letters, digits, '.' and '-' are allowed", s, r)
}
}
return nil
}
// SupportedString renders Supported for a log line or an error message.
func SupportedString() string {
out := make([]string, 0, len(versions))
for _, v := range Supported() {
out = append(out, string(v))
}
return strings.Join(out, ", ")
}
@@ -0,0 +1,234 @@
package roomversion
import (
"errors"
"strings"
"testing"
)
func TestGet(t *testing.T) {
t.Run("a supported version", func(t *testing.T) {
rules, err := Get(V11)
if err != nil {
t.Fatalf("Get(11): %v", err)
}
if rules.Version != V11 {
t.Errorf("Version = %q, want 11", rules.Version)
}
if !rules.Supported {
t.Error("the rules we were handed say they are not supported")
}
})
t.Run("a known but unsupported version", func(t *testing.T) {
_, err := Get(V1)
if !errors.Is(err, ErrUnsupportedVersion) {
t.Errorf("Get(1) = %v, want ErrUnsupportedVersion", err)
}
})
t.Run("an unknown version", func(t *testing.T) {
_, err := Get("99")
if !errors.Is(err, ErrUnknownVersion) {
t.Errorf("Get(99) = %v, want ErrUnknownVersion", err)
}
})
}
func TestMustGetPanicsOnAnUnsupportedVersion(t *testing.T) {
defer func() {
if recover() == nil {
t.Error("MustGet did not panic on an unsupported version")
}
}()
MustGet(V1)
}
func TestDefaultIsSupported(t *testing.T) {
if _, err := Get(Default); err != nil {
t.Fatalf("the default room version %q is not usable: %v", Default, err)
}
if !IsSupported(Default) {
t.Errorf("IsSupported(%q) = false", Default)
}
}
func TestSupported(t *testing.T) {
supported := Supported()
if len(supported) == 0 {
t.Fatal("no room versions are supported")
}
for _, v := range supported {
if !IsSupported(v) {
t.Errorf("Supported() lists %q but IsSupported says no", v)
}
}
for _, v := range []Version{V1, V2} {
if IsSupported(v) {
t.Errorf("IsSupported(%q) = true, but the event format for it is not implemented", v)
}
}
if s := SupportedString(); !strings.Contains(s, string(Default)) {
t.Errorf("SupportedString() = %q, want it to include the default", s)
}
}
func TestKnownListsEveryVersion(t *testing.T) {
known := Known()
if len(known) < 12 {
t.Errorf("Known() returned %d versions, want every version from 1 to 12", len(known))
}
// Known hands out a copy, so a caller cannot edit the registry.
known[0].Supported = !known[0].Supported
if Known()[0].Supported == known[0].Supported {
t.Error("Known() returns the registry itself rather than a copy")
}
}
// TestFlagsAreMonotonic checks the flags that a later room version never
// takes back. Each of these was added once and kept, so a version where
// one is off after a version where it is on is a transcription error in
// the table — the kind of mistake that would otherwise only show up as a
// room whose history stops verifying.
func TestFlagsAreMonotonic(t *testing.T) {
flags := map[string]func(Rules) bool{
"StrictCanonicalJSON": func(r Rules) bool { return r.StrictCanonicalJSON },
"EnforceSigningKeyValidity": func(r Rules) bool { return r.EnforceSigningKeyValidity },
"KnockJoinRule": func(r Rules) bool { return r.KnockJoinRule },
"RestrictedJoinRule": func(r Rules) bool { return r.RestrictedJoinRule },
"KnockRestrictedJoinRule": func(r Rules) bool { return r.KnockRestrictedJoinRule },
"RequireIntegerPowerLevels": func(r Rules) bool { return r.RequireIntegerPowerLevels },
"RedactsInContent": func(r Rules) bool { return r.RedactsInContent },
"RedactionKeepsWholeCreateContent": func(r Rules) bool { return r.RedactionKeepsWholeCreateContent },
"RedactionKeepsRedacts": func(r Rules) bool { return r.RedactionKeepsRedacts },
"RedactionKeepsPowerLevelsInvite": func(r Rules) bool { return r.RedactionKeepsPowerLevelsInvite },
"RedactionKeepsThirdPartyInviteSigned": func(r Rules) bool { return r.RedactionKeepsThirdPartyInviteSigned },
"RedactionKeepsJoinAuthorisedViaUsersServer": func(r Rules) bool { return r.RedactionKeepsJoinAuthorisedViaUsersServer },
"RedactionKeepsJoinRulesAllow": func(r Rules) bool { return r.RedactionKeepsJoinRulesAllow },
"RoomIDIsCreateEventID": func(r Rules) bool { return r.RoomIDIsCreateEventID },
"CreateEventExcludedFromAuthEvents": func(r Rules) bool { return r.CreateEventExcludedFromAuthEvents },
"CreatorsHaveInfinitePower": func(r Rules) bool { return r.CreatorsHaveInfinitePower },
"AdditionalCreators": func(r Rules) bool { return r.AdditionalCreators },
}
known := Known()
for name, get := range flags {
seen := false
for _, rules := range known {
if get(rules) {
seen = true
continue
}
if seen {
t.Errorf("%s is on before room version %s and off again at it", name, rules.Version)
break
}
}
if !seen {
t.Errorf("%s is never on in any room version", name)
}
}
}
// TestFlagsThatWereDropped covers the other direction: the four pieces of
// behaviour a later room version took away.
func TestFlagsThatWereDropped(t *testing.T) {
cases := []struct {
name string
get func(Rules) bool
lastOnFor Version
}{
{"AliasesEventAuth", func(r Rules) bool { return r.AliasesEventAuth }, V5},
{"RedactionKeepsAliases", func(r Rules) bool { return r.RedactionKeepsAliases }, V5},
{"CreateEventRequiresCreator", func(r Rules) bool { return r.CreateEventRequiresCreator }, V10},
{"RedactionProtectsLegacyKeys", func(r Rules) bool { return r.RedactionProtectsLegacyKeys }, V10},
}
known := Known()
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
past := false
for _, rules := range known {
want := !past
if got := tc.get(rules); got != want {
t.Errorf("room version %s has %s = %v, want %v", rules.Version, tc.name, got, want)
}
if rules.Version == tc.lastOnFor {
past = true
}
}
})
}
}
func TestEventIDFormatByVersion(t *testing.T) {
cases := map[Version]EventIDFormat{
V1: EventIDFormatExplicit,
V2: EventIDFormatExplicit,
V3: EventIDFormatSHA256,
V4: EventIDFormatSHA256URLSafe,
V11: EventIDFormatSHA256URLSafe,
V12: EventIDFormatSHA256URLSafe,
}
for _, rules := range Known() {
want, ok := cases[rules.Version]
if !ok {
continue
}
if rules.EventIDFormat != want {
t.Errorf("room version %s has event ID format %d, want %d", rules.Version, rules.EventIDFormat, want)
}
}
}
func TestStateResByVersion(t *testing.T) {
cases := map[Version]StateResVersion{
V1: StateResV1,
V2: StateResV2,
V11: StateResV2,
V12: StateResV2Subgraph,
}
for _, rules := range Known() {
want, ok := cases[rules.Version]
if !ok {
continue
}
if rules.StateRes != want {
t.Errorf("room version %s uses state resolution %d, want %d", rules.Version, rules.StateRes, want)
}
}
}
func TestValidate(t *testing.T) {
valid := []string{"1", "11", "12", "org.example.room-version.1", "a.b-c", strings.Repeat("a", 32)}
for _, s := range valid {
if err := Validate(s); err != nil {
t.Errorf("Validate(%q) = %v, want it to be accepted", s, err)
}
}
invalid := map[string]string{
"": "empty",
strings.Repeat("a", 33): "the limit is",
"V11": "contains",
"11_beta": "contains",
"org.example/room": "contains",
"room version 11": "contains",
}
for s, want := range invalid {
err := Validate(s)
if err == nil {
t.Errorf("Validate(%q) was accepted, want an error", s)
continue
}
if !strings.Contains(err.Error(), want) {
t.Errorf("Validate(%q) = %q, want it to mention %q", s, err, want)
}
}
}
func TestRulesString(t *testing.T) {
if got := MustGet(V11).String(); got != "11" {
t.Errorf("String() = %q, want 11", got)
}
}
+195
View File
@@ -0,0 +1,195 @@
package signing
import (
"crypto/ed25519"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"maps"
"strings"
"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
)
// Signatures is the signatures block of a signed JSON object: server name,
// then key ID ("ed25519:abc"), then the unpadded base64 signature.
//
// An object can carry several, and adding one must not invalidate the
// others — which is why signatures, along with unsigned, is left out of the
// bytes a signature covers.
type Signatures map[string]map[string]string
// Get returns one signature, and whether it was there.
func (s Signatures) Get(serverName, keyID string) (string, bool) {
sig, ok := s[serverName][keyID]
return sig, ok
}
// Errors from verification. ErrNoSignature and ErrBadSignature are
// separate because only the second one is evidence of anything: a missing
// signature usually means we have not fetched the right key yet, while a
// wrong one means the event has been tampered with.
var (
// ErrNoSignature means the object carries no signature from the server
// and key we were asked about.
ErrNoSignature = errors.New("no signature from that server and key")
// ErrBadSignature means a signature was present and did not verify.
ErrBadSignature = errors.New("signature does not verify")
)
// SigningBytes returns the bytes a signature over msg covers: the object
// with its signatures and unsigned properties removed, in canonical JSON.
//
// Both ends of a signature have to agree on these bytes exactly, so this
// is the single place the rule is written down.
func SigningBytes(msg []byte) ([]byte, error) {
obj, err := decodeObject(msg)
if err != nil {
return nil, err
}
delete(obj, "signatures")
delete(obj, "unsigned")
return canonicaljson.Marshal(obj)
}
// SignJSON returns msg with a signature from serverName added.
//
// Signatures already on the object are kept; one from the same server and
// key ID is replaced. The result is canonical JSON, so signing is
// idempotent in the sense that matters: signing the output again produces
// the same bytes.
func SignJSON(serverName string, key *KeyPair, msg []byte) ([]byte, error) {
if serverName == "" {
return nil, errors.New("sign JSON: server name must not be empty")
}
obj, err := decodeObject(msg)
if err != nil {
return nil, err
}
existing, err := signaturesOf(obj)
if err != nil {
return nil, err
}
// The signature covers the object without signatures or unsigned, so
// take them out before canonicalising and put them back afterwards.
unsigned, hasUnsigned := obj["unsigned"]
delete(obj, "signatures")
delete(obj, "unsigned")
signed, err := canonicaljson.Marshal(obj)
if err != nil {
return nil, fmt.Errorf("sign JSON: %w", err)
}
sig := ed25519.Sign(key.Private, signed)
if existing == nil {
existing = Signatures{}
}
if existing[serverName] == nil {
existing[serverName] = map[string]string{}
}
existing[serverName][key.KeyID()] = base64.RawStdEncoding.EncodeToString(sig)
obj["signatures"], err = json.Marshal(existing)
if err != nil {
return nil, fmt.Errorf("sign JSON: encode signatures: %w", err)
}
if hasUnsigned {
obj["unsigned"] = unsigned
}
return canonicaljson.Marshal(obj)
}
// VerifyJSON checks msg for a valid signature from serverName made with
// keyID, using the public key given.
//
// keyID is the full identifier including the algorithm, as it appears in
// the signatures block: "ed25519:abc123".
func VerifyJSON(serverName, keyID string, pub ed25519.PublicKey, msg []byte) error {
obj, err := decodeObject(msg)
if err != nil {
return err
}
sigs, err := signaturesOf(obj)
if err != nil {
return err
}
encoded, ok := sigs.Get(serverName, keyID)
if !ok {
return fmt.Errorf("%w: %s %s", ErrNoSignature, serverName, keyID)
}
sig, err := decodeBase64(encoded)
if err != nil {
return fmt.Errorf("%w: %s %s: decode: %w", ErrBadSignature, serverName, keyID, err)
}
if len(sig) != ed25519.SignatureSize {
return fmt.Errorf("%w: %s %s: signature is %d bytes, want %d",
ErrBadSignature, serverName, keyID, len(sig), ed25519.SignatureSize)
}
delete(obj, "signatures")
delete(obj, "unsigned")
signed, err := canonicaljson.Marshal(obj)
if err != nil {
return err
}
if !ed25519.Verify(pub, signed, sig) {
return fmt.Errorf("%w: %s %s", ErrBadSignature, serverName, keyID)
}
return nil
}
// ExtractSignatures returns the signatures block of a signed object. It is
// how a caller finds out which keys it needs before it has any of them.
func ExtractSignatures(msg []byte) (Signatures, error) {
obj, err := decodeObject(msg)
if err != nil {
return nil, err
}
return signaturesOf(obj)
}
// SplitKeyID splits "ed25519:abc123" into its algorithm and identifier.
func SplitKeyID(keyID string) (algorithm, id string, ok bool) {
algorithm, id, ok = strings.Cut(keyID, ":")
if !ok || algorithm == "" || id == "" {
return "", "", false
}
return algorithm, id, true
}
// decodeObject parses a JSON object into its properties, leaving each value
// as the bytes it arrived as.
func decodeObject(msg []byte) (map[string]json.RawMessage, error) {
var obj map[string]json.RawMessage
if err := json.Unmarshal(msg, &obj); err != nil {
return nil, fmt.Errorf("parse signed JSON: %w", err)
}
if obj == nil {
return nil, errors.New("parse signed JSON: value is null, not an object")
}
return obj, nil
}
// signaturesOf decodes the signatures property, which may be absent.
func signaturesOf(obj map[string]json.RawMessage) (Signatures, error) {
raw, ok := obj["signatures"]
if !ok {
return nil, nil
}
var sigs Signatures
if err := json.Unmarshal(raw, &sigs); err != nil {
return nil, fmt.Errorf("parse signatures: %w", err)
}
// Clone so that a caller mutating the result cannot reach back into
// the object we parsed.
out := make(Signatures, len(sigs))
for server, keys := range sigs {
out[server] = maps.Clone(keys)
}
return out, nil
}
+256
View File
@@ -0,0 +1,256 @@
package signing
import (
"crypto/ed25519"
"encoding/base64"
"errors"
"strings"
"testing"
)
// The signing key, server name and key ID from the spec appendices,
// section "Cryptographic Test Vectors". Every implementation that signs
// JSON should produce the signatures below from them.
const (
specSeed = "YJDBA9Xnr2sVqXD9Vj7XVUnmFZcZrlw8Md7kMW+3XA1"
specServerName = "domain"
specKeyID = "1"
)
func specKey(t *testing.T) *KeyPair {
t.Helper()
seed, err := base64.RawStdEncoding.DecodeString(specSeed)
if err != nil {
t.Fatalf("decode the spec seed: %v", err)
}
priv := ed25519.NewKeyFromSeed(seed)
pub, ok := priv.Public().(ed25519.PublicKey)
if !ok {
t.Fatal("unexpected public key type")
}
return &KeyPair{ID: specKeyID, Private: priv, Public: pub}
}
// TestSpecJSONSigningVectors signs the two objects from the spec and
// compares the result byte for byte.
func TestSpecJSONSigningVectors(t *testing.T) {
key := specKey(t)
cases := []struct {
name string
in string
want string
}{
{
name: "empty object",
in: `{}`,
want: `{"signatures":{"domain":{"ed25519:1":"K8280/U9SSy9IVtjBuVeLr+HpOB4BQFWbg+UZaADMtTdGYI7Geitb76LTrr5QV/7Xg4ahLwYGYZzuHGZKM5ZAQ"}}}`,
},
{
name: "object with data",
in: `{"one":1,"two":"Two"}`,
want: `{"one":1,"signatures":{"domain":{"ed25519:1":"KqmLSbO39/Bzb0QIYE82zqLwsA+PDzYIpIRA2sRQ4sL53+sN6/fpNSoqE7BP7vBZhG6kYdD13EIMJpvhJI+6Bw"}},"two":"Two"}`,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, err := SignJSON(specServerName, key, []byte(tc.in))
if err != nil {
t.Fatalf("SignJSON: %v", err)
}
if string(got) != tc.want {
t.Errorf("SignJSON() =\n %s\nwant\n %s", got, tc.want)
}
if err := VerifyJSON(specServerName, "ed25519:"+specKeyID, key.Public, got); err != nil {
t.Errorf("VerifyJSON on our own signature: %v", err)
}
})
}
}
// TestSignatureIgnoresUnsignedAndSignatures is the property that lets a
// server add a signature, or a timestamp under unsigned, to an object
// without invalidating what is already there.
func TestSignatureIgnoresUnsignedAndSignatures(t *testing.T) {
key := specKey(t)
keyID := "ed25519:" + specKeyID
signed, err := SignJSON(specServerName, key, []byte(`{"a":1,"unsigned":{"age":5}}`))
if err != nil {
t.Fatalf("SignJSON: %v", err)
}
if !strings.Contains(string(signed), `"unsigned":{"age":5}`) {
t.Errorf("unsigned was dropped from the signed object: %s", signed)
}
// Rewrite unsigned and add another server's signature; ours must
// still verify.
tampered := strings.Replace(string(signed), `"age":5`, `"age":9999`, 1)
if err := VerifyJSON(specServerName, keyID, key.Public, []byte(tampered)); err != nil {
t.Errorf("changing unsigned broke the signature: %v", err)
}
other, err := Generate("other")
if err != nil {
t.Fatal(err)
}
countersigned, err := SignJSON("second.example", other, signed)
if err != nil {
t.Fatalf("SignJSON for a second server: %v", err)
}
if err := VerifyJSON(specServerName, keyID, key.Public, countersigned); err != nil {
t.Errorf("a second signature broke the first: %v", err)
}
if err := VerifyJSON("second.example", other.KeyID(), other.Public, countersigned); err != nil {
t.Errorf("the second signature does not verify: %v", err)
}
}
// TestSignJSONReplacesOurOwnSignature checks that signing twice with the
// same key leaves one signature, not two entries fighting over the slot.
func TestSignJSONReplacesOurOwnSignature(t *testing.T) {
key := specKey(t)
once, err := SignJSON(specServerName, key, []byte(`{"a":1}`))
if err != nil {
t.Fatal(err)
}
twice, err := SignJSON(specServerName, key, once)
if err != nil {
t.Fatal(err)
}
if string(once) != string(twice) {
t.Errorf("signing twice changed the object:\n %s\n %s", once, twice)
}
}
func TestVerifyJSONRejects(t *testing.T) {
key := specKey(t)
keyID := "ed25519:" + specKeyID
signed, err := SignJSON(specServerName, key, []byte(`{"a":1}`))
if err != nil {
t.Fatal(err)
}
t.Run("content changed", func(t *testing.T) {
tampered := strings.Replace(string(signed), `"a":1`, `"a":2`, 1)
err := VerifyJSON(specServerName, keyID, key.Public, []byte(tampered))
if !errors.Is(err, ErrBadSignature) {
t.Errorf("error = %v, want ErrBadSignature", err)
}
})
t.Run("wrong key", func(t *testing.T) {
other, err := Generate("other")
if err != nil {
t.Fatal(err)
}
err = VerifyJSON(specServerName, keyID, other.Public, signed)
if !errors.Is(err, ErrBadSignature) {
t.Errorf("error = %v, want ErrBadSignature", err)
}
})
t.Run("no signature from that server", func(t *testing.T) {
err := VerifyJSON("somebody.else", keyID, key.Public, signed)
if !errors.Is(err, ErrNoSignature) {
t.Errorf("error = %v, want ErrNoSignature", err)
}
})
t.Run("no signature with that key ID", func(t *testing.T) {
err := VerifyJSON(specServerName, "ed25519:other", key.Public, signed)
if !errors.Is(err, ErrNoSignature) {
t.Errorf("error = %v, want ErrNoSignature", err)
}
})
t.Run("signature is not base64", func(t *testing.T) {
broken := strings.Replace(string(signed), `"ed25519:1":"`, `"ed25519:1":"!!!`, 1)
err := VerifyJSON(specServerName, keyID, key.Public, []byte(broken))
if !errors.Is(err, ErrBadSignature) {
t.Errorf("error = %v, want ErrBadSignature", err)
}
})
t.Run("signature is the wrong length", func(t *testing.T) {
short, err := SignJSON(specServerName, key, []byte(`{"a":1}`))
if err != nil {
t.Fatal(err)
}
broken := strings.Replace(string(short), `"ed25519:1":"`, `"ed25519:1":"AAAA`, 1)
err = VerifyJSON(specServerName, keyID, key.Public, []byte(broken))
if !errors.Is(err, ErrBadSignature) {
t.Errorf("error = %v, want ErrBadSignature", err)
}
})
}
func TestSigningBytes(t *testing.T) {
got, err := SigningBytes([]byte(`{"b":2,"signatures":{"x":{}},"unsigned":{"age":1},"a":1}`))
if err != nil {
t.Fatalf("SigningBytes: %v", err)
}
if want := `{"a":1,"b":2}`; string(got) != want {
t.Errorf("SigningBytes() = %s, want %s", got, want)
}
}
func TestExtractSignatures(t *testing.T) {
sigs, err := ExtractSignatures([]byte(`{"signatures":{"a.example":{"ed25519:1":"sig"}}}`))
if err != nil {
t.Fatalf("ExtractSignatures: %v", err)
}
if got, ok := sigs.Get("a.example", "ed25519:1"); !ok || got != "sig" {
t.Errorf("Get() = %q, %v; want \"sig\", true", got, ok)
}
if _, ok := sigs.Get("b.example", "ed25519:1"); ok {
t.Error("Get found a signature for a server that has none")
}
none, err := ExtractSignatures([]byte(`{}`))
if err != nil {
t.Fatalf("ExtractSignatures on an unsigned object: %v", err)
}
if none != nil {
t.Errorf("ExtractSignatures() = %v, want nil for an unsigned object", none)
}
}
func TestSplitKeyID(t *testing.T) {
cases := []struct {
in string
algorithm string
id string
ok bool
}{
{"ed25519:abc", "ed25519", "abc", true},
{"ed25519:", "", "", false},
{":abc", "", "", false},
{"ed25519", "", "", false},
{"", "", "", false},
}
for _, tc := range cases {
algorithm, id, ok := SplitKeyID(tc.in)
if algorithm != tc.algorithm || id != tc.id || ok != tc.ok {
t.Errorf("SplitKeyID(%q) = %q, %q, %v; want %q, %q, %v",
tc.in, algorithm, id, ok, tc.algorithm, tc.id, tc.ok)
}
}
}
func TestSignJSONRejectsBadInput(t *testing.T) {
key := specKey(t)
for _, tc := range []struct{ name, in string }{
{"not an object", `[1,2]`},
{"null", `null`},
{"not JSON", `{`},
{"a float somewhere", `{"a":1.5}`},
} {
if _, err := SignJSON(specServerName, key, []byte(tc.in)); err == nil {
t.Errorf("SignJSON(%s) succeeded, want an error", tc.name)
}
}
if _, err := SignJSON("", key, []byte(`{}`)); err == nil {
t.Error("SignJSON with no server name succeeded, want an error")
}
}
+10 -2
View File
@@ -172,8 +172,16 @@ func parseKeyLine(line, path string) (*KeyPair, error) {
return &KeyPair{ID: parts[1], Private: priv, Public: pub}, nil
}
// decodeBase64 accepts both padded and unpadded standard base64, because
// keys get copied between implementations by hand.
// DecodeBase64 decodes a base64 value from the Matrix wire format.
//
// Matrix writes these unpadded, but the spec asks decoders to accept
// padding too wherever they can: keys, hashes and signatures get copied
// between implementations by hand, and rejecting a trailing "=" would turn
// a harmless transcription into a verification failure.
func DecodeBase64(s string) ([]byte, error) {
return decodeBase64(s)
}
func decodeBase64(s string) ([]byte, error) {
if strings.HasSuffix(s, "=") {
return base64.StdEncoding.DecodeString(s)
+245
View File
@@ -0,0 +1,245 @@
package stateres
import (
"container/heap"
"context"
"math"
"slices"
"strings"
"github.com/dariuszjaniak/warp/server/internal/matrix/event"
)
// reverseTopologicalPowerSort orders events earliest to latest, so that
// nothing comes before an event that authorises it, and where there is a
// choice the sender with the most power goes first.
//
// It is Kahn's algorithm with a priority queue: at every step, of the
// events whose auth events have all been placed, the one that sorts
// smallest is taken. Tie-breaking on timestamp and then on event ID makes
// the result identical on every server, which is the whole requirement —
// two servers that ordered these differently would resolve the state
// differently and the room would fork.
func (r *resolver) reverseTopologicalPowerSort(ctx context.Context, events map[string]*event.Event) ([]*event.Event, error) {
// Edges run from an auth event to the event it authorises, counted
// only within the set being sorted: a reference out of the set has
// already happened as far as this ordering goes.
authorises := make(map[string][]string, len(events))
remaining := make(map[string]int, len(events))
for id, e := range events {
for _, authID := range e.AuthEventIDs() {
if _, inSet := events[authID]; !inSet {
continue
}
authorises[authID] = append(authorises[authID], id)
remaining[id]++
}
}
keys := make(map[string]sortKey, len(events))
for id, e := range events {
level, err := r.senderPowerLevel(ctx, e)
if err != nil {
return nil, err
}
keys[id] = sortKey{rank: level, ts: e.OriginServerTS(), id: id}
}
ready := &sortHeap{keys: keys}
for id := range events {
if remaining[id] == 0 {
ready.ids = append(ready.ids, id)
}
}
heap.Init(ready)
out := make([]*event.Event, 0, len(events))
for ready.Len() > 0 {
id := heap.Pop(ready).(string)
out = append(out, events[id])
for _, next := range authorises[id] {
remaining[next]--
if remaining[next] == 0 {
heap.Push(ready, next)
}
}
}
// A cycle in the auth graph would leave events unplaced. It cannot
// happen in a well-formed room, where an event's auth events always
// precede it, but a malicious server can send anything and quietly
// dropping state events would be worse than an arbitrary but stable
// order.
if len(out) < len(events) {
leftover := make([]*event.Event, 0, len(events)-len(out))
for id, e := range events {
if remaining[id] > 0 {
leftover = append(leftover, e)
}
}
sortByKey(leftover, keys)
out = append(out, leftover...)
}
return out, nil
}
// mainlineSort orders events by where they hang off the room's succession
// of power levels events.
//
// The mainline is that succession: the given power levels event, the one
// in its auth events, and so on back to the first one in the room. An
// event's position is how far back along it the event depends, and an
// event depending on an older link sorts first. The effect is that a
// change made under an older set of power levels is applied before one
// made under a newer set, so the newer set has the last word.
func (r *resolver) mainlineSort(ctx context.Context, events map[string]*event.Event, powerLevels *event.Event) ([]*event.Event, error) {
// index maps each power levels event in the mainline to its place,
// 0 being the most recent.
index := map[string]int{}
for current := powerLevels; current != nil; {
if _, loop := index[current.EventID()]; loop {
break // a malformed room; stop rather than walk for ever
}
index[current.EventID()] = len(index)
next, err := r.powerLevelsInAuthEvents(ctx, current)
if err != nil {
return nil, err
}
current = next
}
keys := make(map[string]sortKey, len(events))
positions := map[string]int{}
for id, e := range events {
position, err := r.mainlinePosition(ctx, e, index, positions)
if err != nil {
return nil, err
}
keys[id] = sortKey{rank: int64(position), ts: e.OriginServerTS(), id: id}
}
out := make([]*event.Event, 0, len(events))
for _, e := range events {
out = append(out, e)
}
sortByKey(out, keys)
return out, nil
}
// noMainlinePosition is the spec's "infinity": an event with no power
// levels event behind it at all. It sorts before everything else, because
// it depends on a point in the room older than any power change.
const noMainlinePosition = math.MaxInt
// mainlinePosition returns how far back in the mainline an event depends.
//
// The event itself is not consulted, only the power levels events it
// reaches through its auth events — so a power levels event's own position
// is its predecessor's. That reading never comes up in practice: every
// power levels event is a power event, and power events are all settled in
// the first pass, so none of them reaches this sort.
//
// Each event cites at most one power levels event, so the walk follows a
// single path and every event along it shares the answer.
func (r *resolver) mainlinePosition(ctx context.Context, e *event.Event, index map[string]int, memo map[string]int) (int, error) {
var walked []string
current := e
for {
id := current.EventID()
if position, ok := memo[id]; ok {
return record(walked, position, memo), nil
}
next, err := r.powerLevelsInAuthEvents(ctx, current)
if err != nil {
return 0, err
}
if next == nil {
return record(append(walked, id), noMainlinePosition, memo), nil
}
if position, ok := index[next.EventID()]; ok {
return record(append(walked, id), position, memo), nil
}
walked = append(walked, id)
current = next
}
}
// record caches one position for every event on the path that led to it.
func record(walked []string, position int, memo map[string]int) int {
for _, id := range walked {
memo[id] = position
}
return position
}
// powerLevelsInAuthEvents returns the m.room.power_levels event an event
// cites, if it cites one.
func (r *resolver) powerLevelsInAuthEvents(ctx context.Context, e *event.Event) (*event.Event, error) {
for _, id := range e.AuthEventIDs() {
cited, err := r.get(ctx, id)
if err != nil {
return nil, err
}
if cited == nil {
continue
}
if sk, isState := cited.StateKey(); isState && sk == "" && cited.Type() == event.TypePowerLevels {
return cited, nil
}
}
return nil, nil
}
// sortKey is what both orderings compare.
//
// rank holds the sender's power level for the reverse topological power
// ordering and the mainline position for the mainline ordering. The two
// orderings are otherwise the same comparison, and in both of them a
// higher rank sorts first: more power goes first, and so does a position
// further back in the mainline.
type sortKey struct {
rank int64
ts int64
id string
}
func compareKeys(a, b sortKey) int {
switch {
case a.rank > b.rank:
return -1
case a.rank < b.rank:
return 1
case a.ts < b.ts:
return -1
case a.ts > b.ts:
return 1
default:
// Event IDs are unique, so the order is total: there is never a
// tie left for a server to break its own way.
return strings.Compare(a.id, b.id)
}
}
func sortByKey(events []*event.Event, keys map[string]sortKey) {
slices.SortFunc(events, func(a, b *event.Event) int {
return compareKeys(keys[a.EventID()], keys[b.EventID()])
})
}
// sortHeap is the priority queue Kahn's algorithm picks from.
type sortHeap struct {
ids []string
keys map[string]sortKey
}
func (h *sortHeap) Len() int { return len(h.ids) }
func (h *sortHeap) Less(i, j int) bool { return compareKeys(h.keys[h.ids[i]], h.keys[h.ids[j]]) < 0 }
func (h *sortHeap) Swap(i, j int) { h.ids[i], h.ids[j] = h.ids[j], h.ids[i] }
func (h *sortHeap) Push(x any) { h.ids = append(h.ids, x.(string)) }
func (h *sortHeap) Pop() any {
last := len(h.ids) - 1
out := h.ids[last]
h.ids = h.ids[:last]
return out
}
+609
View File
@@ -0,0 +1,609 @@
// Package stateres implements Matrix state resolution, version 2.
//
// Every event in a room names the events it follows, so a room is a graph
// rather than a line, and two servers can quite legitimately extend it at
// the same time. State resolution is the function that takes the differing
// views of the room's state at those points and produces one answer that
// every server computes identically. It is the reason two people who were
// both made moderator in the same second do not end up with the room in
// two different shapes.
//
// The algorithm is in the room version 2 specification, and room version
// 12 amends it. In outline:
//
// 1. State that every view agrees on is set aside as unconflicted.
// 2. The events in dispute, plus the events that authorise them, are
// sorted so that whoever had the most power goes first.
// 3. They are replayed through the authorisation rules; anything the
// rules refuse is dropped.
// 4. What is left is sorted again, this time by how far back in the
// room's history of power changes it hangs, and replayed.
// 5. The unconflicted state is put back over the top.
//
// Sorting by power first is what makes the result resistant to a
// misbehaving server: an event that grants power can only win if the event
// that granted *it* power already won.
//
// The algorithm is deliberately total — it always produces a state — so
// nothing here rejects an event. Events the rules refuse are simply not
// applied.
package stateres
import (
"context"
"fmt"
"slices"
"strings"
"github.com/dariuszjaniak/warp/server/internal/matrix/auth"
"github.com/dariuszjaniak/warp/server/internal/matrix/event"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
)
// EventProvider supplies the events state resolution needs beyond the
// state sets themselves: the auth chains behind them, and the chain of
// power levels events used for the second sort.
//
// Events this server does not have, and events it rejected, must be left
// out of the result rather than reported as an error. That is not
// leniency: the spec requires rejected auth events to take no part in
// resolution, and leaving them out here is how a caller says so.
type EventProvider interface {
EventsByID(ctx context.Context, ids []string) ([]*event.Event, error)
}
// Resolve resolves a set of room states into one.
//
// stateSets are the differing views of the room's state — in the usual
// case, the state after each of an event's prev_events. The result is the
// state they resolve to.
//
// Resolving fewer than two states is not an error and does no work: there
// is nothing to disagree about.
func Resolve(ctx context.Context, rules roomversion.Rules, stateSets [][]*event.Event, provider EventProvider) (auth.State, error) {
switch len(stateSets) {
case 0:
return auth.State{}, nil
case 1:
return auth.NewState(stateSets[0]), nil
}
r := &resolver{
rules: rules,
provider: provider,
events: map[string]*event.Event{},
missing: map[string]bool{},
powerOf: map[string]int64{},
}
maps := make([]auth.State, len(stateSets))
for i, set := range stateSets {
maps[i] = auth.NewState(set)
for _, e := range set {
r.events[e.EventID()] = e
}
}
return r.resolve(ctx, maps)
}
// Sorted returns a state map as a slice in a deterministic order, for
// storing or comparing.
func Sorted(state auth.State) []*event.Event {
keys := make([]auth.StateKey, 0, len(state))
for k := range state {
keys = append(keys, k)
}
slices.SortFunc(keys, func(a, b auth.StateKey) int {
if a.Type != b.Type {
return strings.Compare(a.Type, b.Type)
}
return strings.Compare(a.StateKey, b.StateKey)
})
out := make([]*event.Event, 0, len(keys))
for _, k := range keys {
out = append(out, state[k])
}
return out
}
// resolver holds the caches one resolution needs. It is single-use.
type resolver struct {
rules roomversion.Rules
provider EventProvider
// events is every event fetched so far, and missing records the IDs
// that resolved to nothing, so that a dangling reference is looked up
// once rather than on every walk that reaches it.
events map[string]*event.Event
missing map[string]bool
// powerOf caches the sender's power level per event, which the first
// sort asks for repeatedly.
powerOf map[string]int64
}
func (r *resolver) resolve(ctx context.Context, stateSets []auth.State) (auth.State, error) {
unconflicted, conflicted := separate(stateSets)
// The full conflicted set is wider than the events actually in
// dispute: it also takes in the auth difference, the events that only
// *some* of the views rely on. Without them, a view could smuggle in a
// power grant by keeping it out of the state it declares.
full := map[string]*event.Event{}
for id, e := range conflicted {
full[id] = e
}
authDiff, err := r.authDifference(ctx, stateSets)
if err != nil {
return nil, err
}
for id, e := range authDiff {
full[id] = e
}
// Room version 12 adds the events that lie between conflicted events
// in the auth graph, which the auth difference can miss. Both this and
// the empty starting state below exist to make state resets rarer.
if r.rules.StateRes == roomversion.StateResV2Subgraph {
subgraph, err := r.conflictedSubgraph(ctx, conflicted)
if err != nil {
return nil, err
}
for id, e := range subgraph {
full[id] = e
}
}
// Step 1: the power events in the full conflicted set, together with
// whatever in their auth chains is also in dispute.
power, err := r.powerEventsToSort(ctx, full)
if err != nil {
return nil, err
}
sortedPower, err := r.reverseTopologicalPowerSort(ctx, power)
if err != nil {
return nil, err
}
// Step 2. Room version 12 starts from an empty state rather than the
// unconflicted one: an event then has to be authorised by the events
// it actually cites, which stops an unconflicted event from propping
// up a conflicted one that nothing else supports.
start := unconflicted
if r.rules.StateRes == roomversion.StateResV2Subgraph {
start = auth.State{}
}
partial, err := r.iterativeAuthChecks(ctx, start, sortedPower)
if err != nil {
return nil, err
}
// Step 3: everything else, ordered by where it hangs off the room's
// history of power levels events.
rest := make(map[string]*event.Event, len(full))
for id, e := range full {
if _, done := power[id]; !done {
rest[id] = e
}
}
sortedRest, err := r.mainlineSort(ctx, rest, partial.PowerLevels())
if err != nil {
return nil, err
}
// Step 4.
resolved, err := r.iterativeAuthChecks(ctx, partial, sortedRest)
if err != nil {
return nil, err
}
// Step 5: the unconflicted state was never in doubt, so it wins
// outright — including over anything the replay above put in its place.
for k, e := range unconflicted {
resolved[k] = e
}
return resolved, nil
}
// separate splits the state sets into the state they all agree on and the
// state they do not.
//
// A key missing from one view counts as a disagreement, not as a blank:
// one server thinking there is no topic and another thinking there is one
// is exactly the kind of conflict this algorithm exists to settle.
func separate(stateSets []auth.State) (auth.State, map[string]*event.Event) {
unconflicted := auth.State{}
conflicted := map[string]*event.Event{}
keys := map[auth.StateKey]bool{}
for _, set := range stateSets {
for k := range set {
keys[k] = true
}
}
for k := range keys {
var distinct []*event.Event
seen := map[string]bool{}
absent := false
for _, set := range stateSets {
e, ok := set[k]
if !ok {
absent = true
continue
}
if !seen[e.EventID()] {
seen[e.EventID()] = true
distinct = append(distinct, e)
}
}
if len(distinct) == 1 && !absent {
unconflicted[k] = distinct[0]
continue
}
for _, e := range distinct {
conflicted[e.EventID()] = e
}
}
return unconflicted, conflicted
}
// authDifference returns the events that are in some views' auth chains but
// not in all of them.
func (r *resolver) authDifference(ctx context.Context, stateSets []auth.State) (map[string]*event.Event, error) {
chains := make([]map[string]bool, len(stateSets))
for i, set := range stateSets {
ids := make([]string, 0, len(set))
for _, e := range set {
ids = append(ids, e.EventID())
}
chain, err := r.authChain(ctx, ids)
if err != nil {
return nil, err
}
chains[i] = chain
}
diff := map[string]*event.Event{}
for i, chain := range chains {
for id := range chain {
inAll := true
for j, other := range chains {
if i != j && !other[id] {
inAll = false
break
}
}
if inAll {
continue
}
if e := r.events[id]; e != nil {
diff[id] = e
}
}
}
return diff, nil
}
// authChain returns the transitive closure of the auth_events links from
// the given events, not including the events themselves.
func (r *resolver) authChain(ctx context.Context, ids []string) (map[string]bool, error) {
chain := map[string]bool{}
queue := make([]string, 0, len(ids))
for _, id := range ids {
e, err := r.get(ctx, id)
if err != nil {
return nil, err
}
if e != nil {
queue = append(queue, e.AuthEventIDs()...)
}
}
for len(queue) > 0 {
id := queue[len(queue)-1]
queue = queue[:len(queue)-1]
if chain[id] {
continue
}
e, err := r.get(ctx, id)
if err != nil {
return nil, err
}
if e == nil {
// A reference we cannot follow. It is not an error: the event
// may have been rejected, or we may simply never have seen it.
continue
}
chain[id] = true
queue = append(queue, e.AuthEventIDs()...)
}
return chain, nil
}
// conflictedSubgraph returns the events that lie on an auth_events path
// between two conflicted events, which room version 12 adds to the full
// conflicted set.
//
// It is computed in two passes: walk forwards from the conflicted events
// to find everything they reach, recording the edges in reverse, then walk
// those edges backwards from the conflicted events. A node the second walk
// reaches is one that both descends from a conflicted event and leads to
// one, which is what being on a path between two of them means.
func (r *resolver) conflictedSubgraph(ctx context.Context, conflicted map[string]*event.Event) (map[string]*event.Event, error) {
parents := map[string][]string{}
seen := map[string]bool{}
queue := make([]string, 0, len(conflicted))
for id := range conflicted {
queue = append(queue, id)
seen[id] = true
}
for len(queue) > 0 {
id := queue[len(queue)-1]
queue = queue[:len(queue)-1]
e, err := r.get(ctx, id)
if err != nil {
return nil, err
}
if e == nil {
continue
}
for _, child := range e.AuthEventIDs() {
parents[child] = append(parents[child], id)
if !seen[child] {
seen[child] = true
queue = append(queue, child)
}
}
}
subgraph := map[string]*event.Event{}
visited := map[string]bool{}
queue = queue[:0]
for id := range conflicted {
queue = append(queue, id)
visited[id] = true
}
for len(queue) > 0 {
id := queue[len(queue)-1]
queue = queue[:len(queue)-1]
if e := r.events[id]; e != nil {
subgraph[id] = e
}
for _, parent := range parents[id] {
if !visited[parent] {
visited[parent] = true
queue = append(queue, parent)
}
}
}
return subgraph, nil
}
// powerEventsToSort returns the events the first pass sorts and replays:
// the power events in the full conflicted set, plus anything in their auth
// chains that is also in the full conflicted set.
func (r *resolver) powerEventsToSort(ctx context.Context, full map[string]*event.Event) (map[string]*event.Event, error) {
out := map[string]*event.Event{}
for id, e := range full {
if !isPowerEvent(e) {
continue
}
out[id] = e
chain, err := r.authChain(ctx, []string{id})
if err != nil {
return nil, err
}
for chainID := range chain {
if e, ok := full[chainID]; ok {
out[chainID] = e
}
}
}
return out, nil
}
// isPowerEvent reports whether an event could take away somebody's ability
// to do something in the room, which is what makes it worth sorting first.
func isPowerEvent(e *event.Event) bool {
switch e.Type() {
case event.TypePowerLevels, event.TypeJoinRules:
_, isState := e.StateKey()
return isState
case event.TypeMember:
sk, ok := e.StateKey()
if !ok || sk == e.Sender() {
// Leaving or being banned under your own name takes nothing
// from anybody else.
return false
}
membership, err := e.Membership()
if err != nil {
return false
}
return membership == event.MembershipLeave || membership == event.MembershipBan
default:
return false
}
}
// iterativeAuthChecks replays events through the authorisation rules,
// applying the ones the rules allow and ignoring the rest.
//
// The state each event is judged against is its own auth_events, overlaid
// with whatever the replay has settled so far for the keys the rules will
// read. So an event is authorised by the room as it is being rebuilt,
// falling back to the events it cited for anything not yet decided.
func (r *resolver) iterativeAuthChecks(ctx context.Context, base auth.State, events []*event.Event) (auth.State, error) {
state := make(auth.State, len(base))
for k, e := range base {
state[k] = e
}
for _, e := range events {
checkState := auth.State{}
for _, id := range e.AuthEventIDs() {
cited, err := r.get(ctx, id)
if err != nil {
return nil, err
}
if cited == nil {
// Either we do not have it or we rejected it. A rejected
// auth event must not be used, and this is where that
// happens.
continue
}
if sk, ok := cited.StateKey(); ok {
checkState[auth.StateKey{Type: cited.Type(), StateKey: sk}] = cited
}
}
keys, err := auth.AuthEventKeys(e)
if err != nil {
// A malformed event cannot be authorised, which in this
// algorithm means it is skipped rather than fatal.
continue
}
for _, k := range keys {
if current, ok := state[k]; ok {
checkState[k] = current
}
}
// From room version 12 the create event is in neither the event's
// auth_events nor the keys above, but the rules still need it: it
// says who the creators are, and the room ID is checked against
// it. Every other version has it from auth_events already.
if checkState.Create() == nil {
create, err := r.createEvent(ctx, e)
if err != nil {
return nil, err
}
if create != nil {
checkState[auth.StateKey{Type: event.TypeCreate}] = create
}
}
if err := auth.Check(ctx, e, checkState, intakeVerified{}); err != nil {
continue
}
if sk, ok := e.StateKey(); ok {
state[auth.StateKey{Type: e.Type(), StateKey: sk}] = e
}
}
return state, nil
}
// intakeVerified stands in for the signature check the restricted join
// rule asks for.
//
// Every event that reaches state resolution was accepted into the room
// first, and that is where its signatures — including the one from the
// server authorising a restricted join — were checked. Checking it again
// here would mean fetching another server's keys in the middle of an
// algorithm that has to be deterministic, run identically on every server
// and return without doing any I/O it can avoid. Synapse draws the line in
// the same place.
//
// The consequence is that intake must not be skipped: an event inserted
// into storage without a signature check would be authorised here as
// though it had one.
type intakeVerified struct{}
func (intakeVerified) VerifyEventSignature(context.Context, string, *event.Event) error { return nil }
// createEvent returns the room's m.room.create event, as seen from one
// event in it.
//
// Up to room version 11 every event cites it in auth_events. From version
// 12 it is deliberately left out, because the room ID is its event ID with
// a different sigil — so it is looked up from there instead, which is the
// same guarantee by a shorter route.
func (r *resolver) createEvent(ctx context.Context, e *event.Event) (*event.Event, error) {
for _, id := range e.AuthEventIDs() {
cited, err := r.get(ctx, id)
if err != nil {
return nil, err
}
if cited == nil {
continue
}
if sk, isState := cited.StateKey(); isState && sk == "" && cited.Type() == event.TypeCreate {
return cited, nil
}
}
if !r.rules.RoomIDIsCreateEventID {
return nil, nil
}
roomID := e.RoomID()
if len(roomID) < 2 || roomID[0] != '!' {
return nil, nil
}
return r.get(ctx, "$"+roomID[1:])
}
// get returns an event by ID, from the cache or from the provider.
func (r *resolver) get(ctx context.Context, id string) (*event.Event, error) {
if e, ok := r.events[id]; ok {
return e, nil
}
if r.missing[id] {
return nil, nil
}
fetched, err := r.provider.EventsByID(ctx, []string{id})
if err != nil {
return nil, fmt.Errorf("state resolution: fetch %s: %w", id, err)
}
for _, e := range fetched {
r.events[e.EventID()] = e
}
if e, ok := r.events[id]; ok {
return e, nil
}
r.missing[id] = true
return nil, nil
}
// senderPowerLevel returns the power level of an event's sender, judged
// from the event's own auth_events rather than from the room as it is now.
//
// Judging it from the event's own view is the point: it is what the sender
// could do at the time, which is the only thing every server can agree on
// after the fact.
func (r *resolver) senderPowerLevel(ctx context.Context, e *event.Event) (int64, error) {
if level, ok := r.powerOf[e.EventID()]; ok {
return level, nil
}
var plEvent *event.Event
for _, id := range e.AuthEventIDs() {
cited, err := r.get(ctx, id)
if err != nil {
return 0, err
}
if cited == nil {
continue
}
if sk, isState := cited.StateKey(); isState && sk == "" && cited.Type() == event.TypePowerLevels {
plEvent = cited
break
}
}
createEvent, err := r.createEvent(ctx, e)
if err != nil {
return 0, err
}
level := int64(0)
if createEvent != nil || plEvent != nil {
pl, err := auth.ResolvePowerLevels(plEvent, createEvent, r.rules)
if err != nil {
// A power levels event we cannot read gives its sender
// nothing, which keeps the sort total.
level = 0
} else {
level = pl.UserLevel(e.Sender())
}
}
r.powerOf[e.EventID()] = level
return level, nil
}
@@ -0,0 +1,746 @@
package stateres
import (
"context"
"encoding/json"
"fmt"
"strings"
"testing"
"time"
"github.com/dariuszjaniak/warp/server/internal/matrix/auth"
"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
"github.com/dariuszjaniak/warp/server/internal/matrix/event"
"github.com/dariuszjaniak/warp/server/internal/matrix/roomversion"
"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
)
const (
alice = "@alice:example.com"
bob = "@bob:example.com"
charlie = "@charlie:example.com"
evelyn = "@evelyn:example.com"
zara = "@zara:example.com"
)
func ptr[T any](v T) *T { return &v }
// spec describes one event to put in the graph. Events are identified by a
// short name so that a test reads like the scenario it describes.
type spec struct {
name string
sender string
typ string
stateKey *string
content map[string]any
}
// dag builds a room whose shape the test controls, and serves it back as
// an EventProvider.
//
// The model is the one Synapse's state resolution tests use, which is also
// where several of the scenarios below come from: events are listed with
// the paths that connect them, each event's auth_events are taken from the
// state resolved at its own prev_events, and timestamps increase in
// declaration order so that no outcome rests on an event ID tie-break.
type dag struct {
t *testing.T
rules roomversion.Rules
key *signing.KeyPair
roomID string
byName map[string]*event.Event
byID map[string]*event.Event
after map[string]auth.State
depth map[string]int64
clock int64
}
func (d *dag) EventsByID(_ context.Context, ids []string) ([]*event.Event, error) {
out := make([]*event.Event, 0, len(ids))
for _, id := range ids {
if e := d.byID[id]; e != nil {
out = append(out, e)
}
}
return out, nil
}
// newDAG starts a room: Alice creates it, joins, takes power level 100,
// makes it public, and Bob, Charlie, Evelyn and Zara join. Everything
// after that is up to the test.
func newDAG(t *testing.T, v roomversion.Version) *dag {
t.Helper()
key, err := signing.Generate("test")
if err != nil {
t.Fatalf("generate a signing key: %v", err)
}
d := &dag{
t: t,
rules: roomversion.MustGet(v),
key: key,
byName: map[string]*event.Event{},
byID: map[string]*event.Event{},
after: map[string]auth.State{},
depth: map[string]int64{},
clock: 1700000000000,
}
if !d.rules.RoomIDIsCreateEventID {
d.roomID = "!room:example.com"
}
create := map[string]any{"room_version": string(v)}
if d.rules.CreateEventRequiresCreator {
create["creator"] = alice
}
join := map[string]any{"membership": "join"}
initial := []spec{
{name: "CREATE", sender: alice, typ: event.TypeCreate, stateKey: ptr(""), content: create},
{name: "IMA", sender: alice, typ: event.TypeMember, stateKey: ptr(alice), content: join},
{name: "IPOWER", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100}}},
{name: "IJR", sender: alice, typ: event.TypeJoinRules, stateKey: ptr(""),
content: map[string]any{"join_rule": auth.JoinRulePublic}},
{name: "IMB", sender: bob, typ: event.TypeMember, stateKey: ptr(bob), content: join},
{name: "IMC", sender: charlie, typ: event.TypeMember, stateKey: ptr(charlie), content: join},
{name: "IMZ", sender: zara, typ: event.TypeMember, stateKey: ptr(zara), content: join},
{name: "START", sender: zara, typ: "m.room.message", content: map[string]any{}},
}
// A linear chain: each initial event follows the one before it.
edges := make([][]string, 0, len(initial))
for i := 1; i < len(initial); i++ {
edges = append(edges, []string{initial[i].name, initial[i-1].name})
}
d.build(initial, edges)
return d
}
// build adds events to the graph.
//
// edges are paths listed newest first, so ["END", "MB", "PA"] means PA is
// followed by MB, which is followed by END. specs must be in dependency
// order; the one place that is not obvious, build says so.
func (d *dag) build(specs []spec, edges [][]string) {
d.t.Helper()
prevs := map[string][]string{}
for _, path := range edges {
for i := 0; i < len(path)-1; i++ {
child, parent := path[i], path[i+1]
if !contains(prevs[child], parent) {
prevs[child] = append(prevs[child], parent)
}
}
}
for _, s := range specs {
state := auth.State{}
var prevIDs []string
var depth int64
parents := prevs[s.name]
stateSets := make([][]*event.Event, 0, len(parents))
for _, parent := range parents {
e, ok := d.byName[parent]
if !ok {
d.t.Fatalf("%s names %s as a previous event, but %s has not been built yet",
s.name, parent, parent)
}
prevIDs = append(prevIDs, e.EventID())
if dep := d.depth[parent]; dep > depth {
depth = dep
}
var set []*event.Event
for _, stateEvent := range d.after[parent] {
set = append(set, stateEvent)
}
stateSets = append(stateSets, set)
}
switch len(stateSets) {
case 0:
case 1:
state = d.after[parents[0]]
default:
// A fork's own events have to be authorised against something,
// and state resolution is what the room's own servers would
// use here too.
resolved, err := Resolve(context.Background(), d.rules, stateSets, d)
if err != nil {
d.t.Fatalf("resolve the state before %s: %v", s.name, err)
}
state = resolved
}
content := mustJSON(d.t, s.content)
selected, err := auth.SelectAuthEvents(d.rules, s.typ, s.sender, s.stateKey, content, state)
if err != nil {
d.t.Fatalf("select auth events for %s: %v", s.name, err)
}
authIDs := make([]string, 0, len(selected))
for _, e := range selected {
authIDs = append(authIDs, e.EventID())
}
b := event.Builder{
Type: s.typ,
Sender: s.sender,
StateKey: s.stateKey,
Content: content,
PrevEvents: prevIDs,
AuthEvents: authIDs,
Depth: depth + 1,
}
derivesRoomID := d.rules.RoomIDIsCreateEventID && s.typ == event.TypeCreate
if !derivesRoomID {
b.RoomID = d.roomID
}
d.clock++
e, err := b.Build(time.UnixMilli(d.clock), d.key, d.rules)
if err != nil {
d.t.Fatalf("build %s: %v", s.name, err)
}
if s.typ == event.TypeCreate && d.rules.RoomIDIsCreateEventID {
if d.roomID, err = event.RoomIDOfCreateEvent(e); err != nil {
d.t.Fatalf("derive the room ID: %v", err)
}
}
d.byName[s.name] = e
d.byID[e.EventID()] = e
d.depth[s.name] = depth + 1
next := make(auth.State, len(state)+1)
for k, v := range state {
next[k] = v
}
if sk, ok := e.StateKey(); ok {
next[auth.StateKey{Type: e.Type(), StateKey: sk}] = e
}
d.after[s.name] = next
}
}
// resolveAt resolves the states after each of a node's previous events,
// which is the state that node would be sent in.
func (d *dag) resolveAt(name string) auth.State {
d.t.Helper()
e, ok := d.byName[name]
if !ok {
d.t.Fatalf("no event named %s", name)
}
var stateSets [][]*event.Event
for _, prevID := range e.PrevEventIDs() {
prevName := d.nameOf(prevID)
var set []*event.Event
for _, stateEvent := range d.after[prevName] {
set = append(set, stateEvent)
}
stateSets = append(stateSets, set)
}
resolved, err := Resolve(context.Background(), d.rules, stateSets, d)
if err != nil {
d.t.Fatalf("resolve at %s: %v", name, err)
}
return resolved
}
func (d *dag) nameOf(eventID string) string {
for name, e := range d.byName {
if e.EventID() == eventID {
return name
}
}
d.t.Fatalf("no name for event %s", eventID)
return ""
}
// wantState asserts which named event holds each of the given state keys.
func (d *dag) wantState(state auth.State, want map[auth.StateKey]string) {
d.t.Helper()
for key, wantName := range want {
got, ok := state[key]
if !ok {
d.t.Errorf("(%s, %q) is missing from the resolved state, want %s", key.Type, key.StateKey, wantName)
continue
}
if gotName := d.nameOf(got.EventID()); gotName != wantName {
d.t.Errorf("(%s, %q) resolved to %s, want %s", key.Type, key.StateKey, gotName, wantName)
}
}
}
func mustJSON(t *testing.T, v any) json.RawMessage {
t.Helper()
raw, err := canonicaljson.Marshal(v)
if err != nil {
t.Fatalf("encode test content: %v", err)
}
return raw
}
func contains(s []string, v string) bool {
for _, x := range s {
if x == v {
return true
}
}
return false
}
var (
powerKey = auth.StateKey{Type: event.TypePowerLevels}
joinRulesKey = auth.StateKey{Type: event.TypeJoinRules}
topicKey = auth.StateKey{Type: "m.room.topic"}
)
func memberKey(userID string) auth.StateKey {
return auth.StateKey{Type: event.TypeMember, StateKey: userID}
}
// --- Trivial cases ---
func TestResolveFewerThanTwoStateSets(t *testing.T) {
d := newDAG(t, roomversion.V11)
resolved, err := Resolve(context.Background(), d.rules, nil, d)
if err != nil {
t.Fatalf("Resolve with no state sets: %v", err)
}
if len(resolved) != 0 {
t.Errorf("resolving nothing gave %d events, want none", len(resolved))
}
var one []*event.Event
for _, e := range d.after["START"] {
one = append(one, e)
}
resolved, err = Resolve(context.Background(), d.rules, [][]*event.Event{one}, d)
if err != nil {
t.Fatalf("Resolve with one state set: %v", err)
}
if len(resolved) != len(one) {
t.Errorf("resolving one state set gave %d events, want %d", len(resolved), len(one))
}
}
func TestResolveIdenticalStateSets(t *testing.T) {
d := newDAG(t, roomversion.V11)
var set []*event.Event
for _, e := range d.after["START"] {
set = append(set, e)
}
resolved, err := Resolve(context.Background(), d.rules, [][]*event.Event{set, set}, d)
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if len(resolved) != len(set) {
t.Fatalf("resolved %d events, want %d", len(resolved), len(set))
}
for _, e := range set {
sk, _ := e.StateKey()
if got := resolved[auth.StateKey{Type: e.Type(), StateKey: sk}]; got != e {
t.Errorf("(%s, %q) changed, though nothing was in conflict", e.Type(), sk)
}
}
}
// --- Scenarios ported from Synapse's state resolution tests ---
// TestBanBeatsPowerLevels: Alice bans Bob on one fork while Bob sends a
// power levels event on the other. Bob's event loses, because by the time
// it is replayed he is banned.
func TestBanBeatsPowerLevels(t *testing.T) {
d := newDAG(t, roomversion.V11)
d.build([]spec{
{name: "PA", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100, bob: 50}}},
{name: "MA", sender: alice, typ: event.TypeMember, stateKey: ptr(alice),
content: map[string]any{"membership": "join"}},
{name: "MB", sender: alice, typ: event.TypeMember, stateKey: ptr(bob),
content: map[string]any{"membership": "ban"}},
{name: "PB", sender: bob, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100, bob: 50}}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "MB", "MA", "PA", "START"},
{"END", "PB", "PA"},
})
d.wantState(d.resolveAt("END"), map[auth.StateKey]string{
powerKey: "PA",
memberKey(alice): "MA",
memberKey(bob): "MB",
})
}
// TestJoinRuleEvasion: Alice makes the room private on one fork while
// Evelyn joins on the other. Evelyn's join loses, so closing a room cannot
// be dodged by joining through an older part of the graph.
func TestJoinRuleEvasion(t *testing.T) {
d := newDAG(t, roomversion.V11)
d.build([]spec{
{name: "JR", sender: alice, typ: event.TypeJoinRules, stateKey: ptr(""),
content: map[string]any{"join_rule": auth.JoinRulePrivate}},
{name: "ME", sender: evelyn, typ: event.TypeMember, stateKey: ptr(evelyn),
content: map[string]any{"membership": "join"}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "JR", "START"},
{"END", "ME", "START"},
})
resolved := d.resolveAt("END")
d.wantState(resolved, map[auth.StateKey]string{joinRulesKey: "JR"})
if _, ok := resolved[memberKey(evelyn)]; ok {
t.Error("Evelyn's join survived a join rule change to private")
}
}
// TestTopicAndPowerLevelFork: Alice demotes Bob on one fork; on the other,
// Bob promotes himself back and sets the topic. Alice's demotion wins
// because it was made under power levels Bob's events were not.
func TestTopicAndPowerLevelFork(t *testing.T) {
d := newDAG(t, roomversion.V11)
d.build([]spec{
{name: "T1", sender: alice, typ: "m.room.topic", stateKey: ptr(""), content: map[string]any{}},
{name: "PA1", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100, bob: 50}}},
{name: "T2", sender: alice, typ: "m.room.topic", stateKey: ptr(""), content: map[string]any{}},
{name: "PA2", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100, bob: 0}}},
{name: "PB", sender: bob, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100, bob: 50}}},
{name: "T3", sender: bob, typ: "m.room.topic", stateKey: ptr(""), content: map[string]any{}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "PA2", "T2", "PA1", "T1", "START"},
{"END", "T3", "PB", "PA1"},
})
d.wantState(d.resolveAt("END"), map[auth.StateKey]string{
powerKey: "PA2",
topicKey: "T2",
})
}
// TestTopicReset is the known rough edge of state resolution v2, kept here
// because it is behaviour rather than a bug to fix: a fork from before a
// topic was set, merged after the author was banned, loses the topic. The
// ban stands and the power levels stand; only the topic reverts.
func TestTopicReset(t *testing.T) {
d := newDAG(t, roomversion.V11)
d.build([]spec{
{name: "T1", sender: alice, typ: "m.room.topic", stateKey: ptr(""), content: map[string]any{}},
{name: "PA", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100, bob: 50}}},
{name: "T2", sender: bob, typ: "m.room.topic", stateKey: ptr(""), content: map[string]any{}},
{name: "MB", sender: alice, typ: event.TypeMember, stateKey: ptr(bob),
content: map[string]any{"membership": "ban"}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "MB", "T2", "PA", "T1", "START"},
{"END", "T1"},
})
d.wantState(d.resolveAt("END"), map[auth.StateKey]string{
topicKey: "T1",
memberKey(bob): "MB",
powerKey: "PA",
})
}
// --- Properties that must hold whatever the scenario ---
// TestResolutionDoesNotDependOnStateSetOrder is the property that makes
// the algorithm usable at all: two servers that hold the same views in a
// different order must reach the same state.
func TestResolutionDoesNotDependOnStateSetOrder(t *testing.T) {
for _, v := range []roomversion.Version{roomversion.V11, roomversion.V12} {
t.Run(string(v), func(t *testing.T) {
d := newDAG(t, v)
d.build([]spec{
{name: "PA", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{bob: 50}}},
{name: "T1", sender: bob, typ: "m.room.topic", stateKey: ptr(""),
content: map[string]any{"topic": "one"}},
{name: "T2", sender: alice, typ: "m.room.topic", stateKey: ptr(""),
content: map[string]any{"topic": "two"}},
{name: "MC", sender: alice, typ: event.TypeMember, stateKey: ptr(charlie),
content: map[string]any{"membership": "ban"}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "T1", "PA", "START"},
{"END", "MC", "T2", "START"},
})
sets := func() [][]*event.Event {
var out [][]*event.Event
for _, name := range []string{"T1", "MC"} {
var set []*event.Event
for _, e := range d.after[name] {
set = append(set, e)
}
out = append(out, set)
}
return out
}
forwards := sets()
backwards := [][]*event.Event{forwards[1], forwards[0]}
a, err := Resolve(context.Background(), d.rules, forwards, d)
if err != nil {
t.Fatal(err)
}
b, err := Resolve(context.Background(), d.rules, backwards, d)
if err != nil {
t.Fatal(err)
}
if got, want := describe(d, a), describe(d, b); got != want {
t.Errorf("the order of the state sets changed the result:\n %s\n %s", got, want)
}
})
}
}
// TestResolutionIsIdempotent: resolving a state against itself is the
// identity, so repeating the work cannot drift.
func TestResolutionIsIdempotent(t *testing.T) {
d := newDAG(t, roomversion.V11)
d.build([]spec{
{name: "PA", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100, bob: 50}}},
{name: "MB", sender: alice, typ: event.TypeMember, stateKey: ptr(bob),
content: map[string]any{"membership": "ban"}},
{name: "T1", sender: alice, typ: "m.room.topic", stateKey: ptr(""), content: map[string]any{}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "MB", "PA", "START"},
{"END", "T1", "PA"},
})
once := d.resolveAt("END")
var set []*event.Event
for _, e := range once {
set = append(set, e)
}
twice, err := Resolve(context.Background(), d.rules, [][]*event.Event{set, set}, d)
if err != nil {
t.Fatal(err)
}
if got, want := describe(d, twice), describe(d, once); got != want {
t.Errorf("resolving the result again changed it:\n %s\n %s", got, want)
}
}
// TestUnconflictedStateAlwaysWins: the last step of the algorithm puts the
// unconflicted state back, so an event nobody disputed cannot be displaced
// by the replay.
func TestUnconflictedStateAlwaysWins(t *testing.T) {
d := newDAG(t, roomversion.V11)
d.build([]spec{
{name: "T1", sender: alice, typ: "m.room.topic", stateKey: ptr(""),
content: map[string]any{"topic": "one"}},
{name: "T2", sender: alice, typ: "m.room.topic", stateKey: ptr(""),
content: map[string]any{"topic": "two"}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "T1", "START"},
{"END", "T2", "START"},
})
resolved := d.resolveAt("END")
// The join rules, the create event and everybody's membership were
// never in dispute and must come through untouched.
for _, name := range []string{"CREATE", "IJR", "IMA", "IMB", "IMC", "IMZ"} {
e := d.byName[name]
sk, _ := e.StateKey()
if got := resolved[auth.StateKey{Type: e.Type(), StateKey: sk}]; got != e {
t.Errorf("%s was displaced, though nothing disputed it", name)
}
}
}
// TestVersion12ResolvesARoom checks the version 12 path end to end: the
// create event is found through the room ID rather than through
// auth_events, and the amended algorithm still settles a conflict.
func TestVersion12ResolvesARoom(t *testing.T) {
d := newDAG(t, roomversion.V12)
d.build([]spec{
{name: "PA", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{bob: 50}}},
{name: "MB", sender: alice, typ: event.TypeMember, stateKey: ptr(bob),
content: map[string]any{"membership": "ban"}},
{name: "PB", sender: bob, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{bob: 50, charlie: 50}}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "MB", "PA", "START"},
{"END", "PB", "PA"},
})
resolved := d.resolveAt("END")
d.wantState(resolved, map[auth.StateKey]string{
powerKey: "PA",
memberKey(bob): "MB",
})
if resolved[auth.StateKey{Type: event.TypeCreate}] == nil {
t.Error("the create event is missing from the resolved state")
}
}
// TestMissingEventsAreSkipped: a reference the provider cannot resolve,
// which over federation means an event we never received or one we
// rejected, must not stop resolution.
func TestMissingEventsAreSkipped(t *testing.T) {
d := newDAG(t, roomversion.V11)
d.build([]spec{
{name: "T1", sender: alice, typ: "m.room.topic", stateKey: ptr(""),
content: map[string]any{"topic": "one"}},
{name: "T2", sender: alice, typ: "m.room.topic", stateKey: ptr(""),
content: map[string]any{"topic": "two"}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "T1", "START"},
{"END", "T2", "START"},
})
var sets [][]*event.Event
for _, name := range []string{"T1", "T2"} {
var set []*event.Event
for _, e := range d.after[name] {
set = append(set, e)
}
sets = append(sets, set)
}
resolved, err := Resolve(context.Background(), d.rules, sets, emptyProvider{})
if err != nil {
t.Fatalf("Resolve with a provider that knows nothing: %v", err)
}
if len(resolved) == 0 {
t.Error("resolution produced no state at all")
}
// The topic was in conflict and its auth events cannot be fetched, so
// neither candidate can be authorised; what matters is that the
// unconflicted state still comes through.
if resolved[joinRulesKey] == nil {
t.Error("the unconflicted join rules were lost")
}
}
type emptyProvider struct{}
func (emptyProvider) EventsByID(context.Context, []string) ([]*event.Event, error) {
return nil, nil
}
func TestProviderErrorsAreReported(t *testing.T) {
d := newDAG(t, roomversion.V11)
// PA replaces IPOWER, so IPOWER is in the auth chain of the state but
// not in the state itself — which is exactly when resolution has to go
// to the provider.
d.build([]spec{
{name: "PA", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100, bob: 50}}},
{name: "T1", sender: alice, typ: "m.room.topic", stateKey: ptr(""),
content: map[string]any{"topic": "one"}},
{name: "T2", sender: bob, typ: "m.room.topic", stateKey: ptr(""),
content: map[string]any{"topic": "two"}},
{name: "END", sender: zara, typ: "m.room.message", content: map[string]any{}},
}, [][]string{
{"END", "T1", "PA", "START"},
{"END", "T2", "PA"},
})
var sets [][]*event.Event
for _, name := range []string{"T1", "T2"} {
var set []*event.Event
for _, e := range d.after[name] {
set = append(set, e)
}
sets = append(sets, set)
}
_, err := Resolve(context.Background(), d.rules, sets, failingProvider{})
if err == nil {
t.Fatal("Resolve succeeded with a provider that always fails")
}
if !strings.Contains(err.Error(), "state resolution") {
t.Errorf("error = %q, want it to say where it came from", err)
}
}
type failingProvider struct{}
func (failingProvider) EventsByID(context.Context, []string) ([]*event.Event, error) {
return nil, fmt.Errorf("the database is on fire")
}
// --- Units ---
func TestIsPowerEvent(t *testing.T) {
d := newDAG(t, roomversion.V11)
d.build([]spec{
{name: "PL", sender: alice, typ: event.TypePowerLevels, stateKey: ptr(""),
content: map[string]any{"users": map[string]any{alice: 100}}},
{name: "JR", sender: alice, typ: event.TypeJoinRules, stateKey: ptr(""),
content: map[string]any{"join_rule": auth.JoinRulePublic}},
{name: "BAN", sender: alice, typ: event.TypeMember, stateKey: ptr(bob),
content: map[string]any{"membership": "ban"}},
{name: "KICK", sender: alice, typ: event.TypeMember, stateKey: ptr(charlie),
content: map[string]any{"membership": "leave"}},
{name: "SELF_LEAVE", sender: charlie, typ: event.TypeMember, stateKey: ptr(charlie),
content: map[string]any{"membership": "leave"}},
{name: "JOIN", sender: evelyn, typ: event.TypeMember, stateKey: ptr(evelyn),
content: map[string]any{"membership": "join"}},
{name: "INVITE", sender: alice, typ: event.TypeMember, stateKey: ptr(evelyn),
content: map[string]any{"membership": "invite"}},
{name: "MESSAGE", sender: alice, typ: "m.room.message", content: map[string]any{}},
{name: "TOPIC", sender: alice, typ: "m.room.topic", stateKey: ptr(""), content: map[string]any{}},
}, [][]string{
{"TOPIC", "MESSAGE", "INVITE", "JOIN", "SELF_LEAVE", "KICK", "BAN", "JR", "PL", "START"},
})
want := map[string]bool{
"PL": true, "JR": true, "BAN": true, "KICK": true,
"SELF_LEAVE": false, "JOIN": false, "INVITE": false,
"MESSAGE": false, "TOPIC": false,
}
for name, expected := range want {
if got := isPowerEvent(d.byName[name]); got != expected {
t.Errorf("isPowerEvent(%s) = %v, want %v", name, got, expected)
}
}
}
func TestSorted(t *testing.T) {
d := newDAG(t, roomversion.V11)
sorted := Sorted(d.after["START"])
if len(sorted) == 0 {
t.Fatal("Sorted returned nothing")
}
for i := 1; i < len(sorted); i++ {
prev, next := sorted[i-1], sorted[i]
prevKey, _ := prev.StateKey()
nextKey, _ := next.StateKey()
if prev.Type() > next.Type() || (prev.Type() == next.Type() && prevKey > nextKey) {
t.Errorf("Sorted is out of order at %d: (%s, %q) before (%s, %q)",
i, prev.Type(), prevKey, next.Type(), nextKey)
}
}
}
// describe renders a resolved state as a stable string for comparison.
func describe(d *dag, state auth.State) string {
var parts []string
for _, e := range Sorted(state) {
sk, _ := e.StateKey()
parts = append(parts, fmt.Sprintf("(%s,%s)=%s", e.Type(), sk, d.nameOf(e.EventID())))
}
return strings.Join(parts, " ")
}