Lessons Learned from Three Years of Applied Research

• Not because I'm that interesting, but because y'all are interesting.
• Have already had some great conversations around topics of mutual interest:
  • goal of the intro is to plant seeds for future conversations
• So I'm gonna go fast!

• Computer Science, Virtual Reality, Procedural Content Generation focus
• academic research didn't align with tool-building focus

• One of my formative experiences was working for a company now called Cognitect, that stewards the Clojure language and Datomic

• Real-time collaboration
• Knowledge representation
• Tools for Thought

• Croquet - Fission - Daylight Tablet - DXOS.

• Whether I'm using a shovel or a computational notebook, you are thinking
• False dichotomy between "thinking tools" and "other tools"
• Denigrates tools that aren't considered "thinking tools"

Drop the emphasis on "thinking", instead focus on "tools"

The Computer Scientist as Toolsmith
- Fred Brooks, Turing award winner, acceptance speech
- "Computer Science is the handmaiden of the sciences"

In a word the computer scientist is a toolsmith-no more, but no less. It is an honorable calling.
If we perceive our role aright, we then see more clearly the proper criterion for success: a toolmaker succeeds as, and only as, the users of his tool succeed with his aid. However shining the blade, however jeweled the hilt, however perfect the heft, a sword is tested only by cutting.
That swordsmith is successful whose clients die of old age.

If we want to scale up human reasoning, how do we make tools that assist with that process?

• "Mediums" offer a clue - mediums are generic patterns that show up all over the place
• Elicit, right now, presents itself to the user as a table. Tables, as a medium, have certain affordances.
  • (It's not a spreadsheet because it's not reactively computational)

What if you utilized a different medium?

• A tried-and-true medium is the research notebook.
  • I keep one.
  • What if you equipped AI agents to record notes on papers in a notebook as well as in the tabular format.
    • Changing the medium changes the mode of interaction
  • Show my tools -> mediums graphic

One thing you see a lot in the future of computing and in the AI space right now is cool demos.

• On the surface, I'm not against cool demos, but some folks act as if that's the end of the process.
• Others think "let's build a product", extract some revenue, etc

If the goal is truly to scale up better reasoning, you need to figure out how to connect from demos to true progress in the scientific sense of the word, not only to a dominant product in the market.

Ratcheting progress in tools for thought is a really important essay by Andy Matuschak.

• Andy and Michael Nielsen wrote How can we develop transformative tools for thought? by Matuschak and Nielsen
• 4 step process:
  • ...

Real Contexts of Use Are All The Rage

• Bret Victor, the Father of all Demos (haha) has pivoted to building Dynamicland inside of a computational biology research lab at a univeristy
• Andy has scoped down his work on memory systems to equipping specific users with tools to accmplish near-term objectives
• Ink & Switch's Malleable Software group (Geoffrey Litt) is working with research teams collaborating to write academic publications and building a writing environment

Congratulations Elicit, you have real users! Now what?

• Pay attention to metrics that go beyond product usage.