Making schema migrations smoother

TLDR

We make many improvements to schema migration tooling, add support for UUIDv4 and UUIDv7, and more.

Schema migrations

Triplit provides tools that help you update your schema in a way that will:

1. allow clients with the previous version of the schema to continue syncing and
2. maintain the integrity of data in their caches and on the server

The focus of the release this week was making schema migrations even safer in production and lower friction in development. We have made a number of changes, including:

Reworked npx triplit schema push

The the CLI command for updating the schema of a server is now lower friction and just as safe. Triplit allows two types of schema changes:

1. backwards compatible changes like adding optional columns or new collections
2. backwards incompatible changes like renaming or removing columns that do not lead to data loss or corruption on the central server.

Triplit will allow (1) under any circumstances, but (2) previously required a special flag. Now both (1) and (2) are allowed by default, which is ideal in development when you are frequently iterating on your schema. In production we recommend that you use the new --enforceBackwardsCompatibility flag to only allow (1) and prevent (2).

More relaxed connectivity checks

Previously, a Triplit client would not be able to connect to a server if the client's schema had not previously been pushed to the server. We have relaxed this requirement. Now clients can to a server, regardless of any schema incompatibility. This means that in production old clients will more reliably connect to the server and continue, but it increases the possibility that the client may have runtime errors if the server sends data that is not compatible with the client's schema. This further emphasizes the need to make backwards compatible changes to the schema and write client code that is resilient to schema changes, if you plan on updating the schema in production with backwards incompatible changes.

onDatabaseInit hook

As we relax the default checks on schema compatibility, we plan to add hooks that allow you to handle success or error states as you see fit for your application. The first we are exposing is onDatabaseInit, which runs after your client-side database has initialized and before any client operations are run and syncing begins. This hook provides a DB instance and an event which tells you information about the database startup state. The hook is experimental and may change in the future. However, you may add it to your client options under experimental.onDatabaseInit. The following example shows how to use the hook to clear your local database if the schema cannot be migrated:

import { TriplitClient } from '@triplit/client';
import { Schema as S } from '@triplit/client';

const client = new TriplitClient({
  schema: S.Collections({
    // Your schema here
  }),
  experimental: {
    onDatabaseInit: async (db, event) => {
      if (event.type === 'SUCCESS') return;
      if (event.type === 'SCHEMA_UPDATE_FAILED') {
        if (event.change.code === 'EXISTING_DATA_MISMATCH') {
          // clear local database
          await db.clear();
          // retry schema update
          const nextChange = await db.overrideSchema(event.change.newSchema);
          // Schema update succeeded!
          if (nextChange.successful) return;
        }
      }
      // handle other cases...

      // Handle fatal states as you see fit
      telemetry.reportError('database init failed', {
        event,
      });
      throw new Error('Database init failed');
    },
  },
});

This is the beginning of upcoming work which will allow you to handle schema and data migrations in a more granular way.

Updated documentation

We have revamped our guide to updating your schema. It more clearly states how Triplit categorizes schema changes and when you can expect changes to succeed or fails. To reflect the changes in the CLI, we have also updated the triplit schema push documentation.

Support for UUIDv4 and UUIDv7 default values

You can now use UUIDv4 and UUIDv7 as default values for attributes. This can be specified by passing the format option to the Id schema type. For example:

import { Schema as S } from '@triplit/client';

const schema = S.Collections({
  todos: {
    schema: S.Schema({
      id: S.Id(), // defaults to nanoid
      text: S.String(),
    }),
  },
  users: {
    schema: S.Schema({
      id: S.Id({ format: 'uuidv4' }),
      name: S.String(),
    }),
  },
  posts: {
    schema: S.Schema({
      id: S.Id({ format: 'uuidv7' }),
      title: S.String(),
    }),
  },
});

uuidv4 is built-in and does not require any additional dependencies. uuidv7 requires the uuid package to be installed:

npm i uuidv7

Read more in the schemas documentation.

Use the $prev variable in postUpdate permissions to define new invariants

The postUpdate permission runs after a document is updated to confirm that no invariants are violated. It now has access to the previous value of the document via the $prev variable. This allows you to check that the new value is valid in the context of the old value, e.g. that the previous value was unchanged:

import { Schema as S } from '@triplit/client';

const schema = S.Collections({
  todos: {
    schema: S.Schema({
      id: S.Id(),
      text: S.String(),
      authorId: S.String(),
      updatedAt: S.Date(),
      completed: S.Boolean(),
    }),
    permissions: {
      read: {
        filter: [true],
      },
      insert: {
        filter: [true],
      },
      postUpdate: {
        filter: [
          ['authorId', '=', '$prev.authorId'],
          ['updatedAt', '>', '$prev.updatedAt'],
        ],
      },
    },
  },
});

Other improvements

• Fixed an error that could occur when running npx triplit --help
• Fixed an issue where S.Json attributes were throwing errors when using npx triplit schema print and causing erroneous diffs in npx triplit schema diff
• Fixed a corner case where the fetching states from useQuery were not correctly updating
• Fixed an issue where schema issues were not logging in the client console.
• Fixed an issue where callbacks passed to TriplitClient.onConnectionStatusChange were re-mounting without unsubscribing in disconnect/connect cycles.