In order to make progress in a field, one has to discover and disseminate generalizable insights. Andy Matuschak summarized a process for this in Ratcheting progress in tools for thought:

An idealized cycle of activity might involve something like these key steps:

Identifying powerful insights about some subject domain or about cognition in general which might be fruitfully systematized

Building systems which express those insights in their primitives

Observing serious use of those systems in authentic contexts, and of your theoretical insights refracted through them

Distilling generalizable insight from those observations which produce new understanding about the subject domain or cognition, and which permit new, better systems to be built

Disseminating that insight so that others can build on it

My goal is to advance progress for computers as computational medium, so I need to consider this to be the core goal of my work and take seriously this suggested process. These steps should be a guide for the kind of work that I'm doing during my Dedicated work blocks.

Let's interrogate each of these steps a bit:

1. Identify a powerful insight

This step largely is about intuition developed through familiarity. Another way to think of this is hypothesis formation: I believe that X is generally true about systems that have Y characteristics in some situations. It's vague, based on gut, but the insight needs to be sharpened enough so that it can guide the next step of building a system that expresses the insight.

2. Build systems that express the insight

I know how to build systems that are "expressions of technological or market insights, rather than of fundamental insights about a subject domain or about cognition." How does one build systems designed for the asking and answering of questions?

Furthermore, I used the plural "systems" intentionally here. It is probable that in order to distill generalizable insights, more than one system will have to be built.

3. Observe serious use of the systems

This implies that the system that is built is actually useful for solving a real problem. In other words, it has to serve an important function with a context of use. People have to want to use it. Building a system that is not merely a research prototype, but is being used in a way that can lead to deriving generalizable insights is challenging and requires a tool-maker mindset.

Beyond usefulness and desirability, "the system has to be shaped in a way which allows you to ask the questions you want to ask." It's not clear to me the characteristics of a system that allow for question-answering. Is it merely analytics instrumentation?

4. Distill generalizable insight

This is the gold nugget. Out of the observation of serious use, some generalizable insight can be distilled. Framing this generalizable insight is a challenge unto itself.

Andy offers a good example from learning science:

Educational design principles are usually heuristic statements of a format such as: “If you want to design intervention X (for the purpose/function Y in context Z), then you are best advised to give that intervention the characteristics A, B, and C (substantive emphasis), and to do that via procedures K, L, and M (procedural emphasis), because of arguments P, Q, and R.”

What might be a good formula for characterizing a generalizable insight for tools for thought?

5. Disseminate the insight

Finally, tell people about it.

This seems to me to be the easiest step, but I'm reminded of the quote attributed to Howard Aiken, himself a pioneer of computing: "Don't worry about people stealing your ideas. If your ideas are any good, you'll have to ram them down people's throats."

Nevertheless, it seems worthwhile to start with the previous four steps and worry about dissemination when there is something worth disseminating.

Discover and disseminate generalizable insights