Effect Days 2024 Talk Track - Cleaned Up

Beyond Apps: The distributed schema problem with @effect/schema

introduce how and why "schemas" figure in the n-tier web architecture: in the db, in the API, and on the Client. We primarily need them for validation, but they can also be used for developer tooling, code generation, etc.

describe the problems with this architecture:

  • offline is hard
  • realtime is hard
  • servers and loss of privacy
  • interop: applications create data (schema) silos
  • you're busy shuffling data and transforming it between the three schemas

introduce the local-first architecture with dxos

  • built in realtime collab
  • built in offline
  • built in privacy
  • exploring direct interop without APIs: need a schema-driven DX

our value hypothesis: the schema of the db IS the schema of the API and the frontend - all state lives in strongly typed, validated, reactive objects. this means radically improved developer velocity and new scenarios possible.

show how we use @effect/schema to make our API type-safe:

  • show guestbook app source
  • show declaration of schema
  • show dx of inferred types
  • show dx of queries, explain how validation happened at runtime
  • show dx of mutations, explain how validation happens during mutations

show how schema is important to interop:

  • composer can see the same data as the guestbook app - show how it's reactive between the apps

  • composer can visualize things in tables by examining the schema

  • show ability to infer columns and display guestbook data

  • show ability to add/edit records and perform validation trivially (reject invalid input - i.e. schema in the database was used to protect guestbook app from invalid changes in composer, no extra effort on either app, no API layer in between because both apps use the database directly.)

    • explain the problems with this:
      • how do we protect from invalid business rules that aren't expressed as simple schema constraints? (maybe we need to help invent a second schema layer that's more like the API schema on top of the Data schema?)
  • show ability to extend schema at runtime without breaking either app: add a column and annotate some user records with custom fields in composer - observe data is retained and guestbook app continues to work

  • explain how composer is extensible and can delegate rendering of arbitrary things to plugins, which can use schema to decide how to render things, or delegate parts of rendering to other plugins through more surfaces - surface nesting ftw.

  • show how schema can be used in AI scenarios:

    • show how we can use schema in LLM prompts for augmenting CRM records with more data
    • concrete example: show how we can do entity extraction from a plain text document and use schema to format a prompt that extracts tasks and due dates from some meeting notes
  • conclude with calls to action about composer and dxos.

    • prior art: smalltalk, hypercard, xanadu, microsoft access, others
    • opportunity: private, open source, developer extensible, local-first knowledge management system
    • opportunity: build next generation apps radically faster with dxos
    • call to review our resolvers/mutation adapters + schema designs
Effect Days 2024 Talk Track - Cleaned Up
Interactive graph
On this page
Effect Days 2024 Talk Track - Cleaned Up
Beyond Apps: The distributed schema problem with @effect/schema