I’ve been voraciously reading about how programming (or, creating an interactive system) can be more accessible. Here’s a summary of what I’ve discovered so far:
An project that aims at “finding a better way for us to interact with computers” by creating a programming environment based around transforming data.
Prototype, experiment, and put stuff in front of users. What you think is intuitive doesn’t always make sense to other people.
A relational database is a good place to start.
Everything can be undone. Allow the user to feel comfortable experimenting.
Choosing the wrong word for a concept can alienate and confuse users.
Natural language programming doesn’t need to be the goal. It can create an uncanny valley where the language is too similar to English (or whatever) and any weirdness in the grammar starts to irk.
Their ‘live document’ style of editor with inline queries went nowhere.
A functional reactive programming language for front-end web development.
- The error messages explain what went wrong and suggest ways to fix the problem. The compiler even catches typos. When Elm fans rave about Elm, it’s usually to rave about the helpfulness of the error messages.
A tool for creating interactive fiction thats lays out individual pages as boxes that you can move around, and draws lines between pages that link to each other.
- A personalized map of your code structure creates a memorable and navigable space.
An early Apple product that allowed non-programmers to create simple stack-based programs using a drag-and-drop WYSIWYG editor and a natural-language-like scripting language.
The key to HyperCard’s success was its “radical simplicity and the resulting explorability” and the devoted community of amateur creators that surrounded it. Various clones have arisen over the years, but none have taken HyperCard’s place as beloved software. They tried to be more powerful but just added too many bells and whistles to comfortably understand all at once.
Use a familiar metaphor that’s easy to reason about.
An educational language that allows children to control a ‘Turtle’ that draws geometric shapes on the screen or a sheet of paper.
Allow the user to bring their existing knowledge about the world to the tool. For Logo, that means kids can imagine themselves as the Turtle moving about in space, and from there they could reason about how to program the Turtle.
Experimentation turns abstractions into intuitions. Iteration is important. The user should be able to take chances, get messy, make mistakes.
“When knowledge can be broken up into ‘mind-sized bites,’ it is more communicable.”
Visual programming languages that transform code into interlocking blocks.
The user can select from a palette of objects instead of staring at an empty prompt.
Blocks break code into meaningful chunks that are easily identifiable and easily swapped out. You don’t have to look up function definitions – the blocks visually show what their inputs and outputs are.
The authors of this thesis used psychology to create a better modeling tool called BCM and tested it against traditional UML and ERD methods.
People do not easily think abstractly and analytically. As much as possible, keep concepts concrete and related to concepts they already know. (e.g., “Person,” “Place,” and “Document” work better than “Category,” “Idea,” or “System.”)
Build new models from existing concepts.
Relationships (links between concepts) are the hardest thing to model and the easiest thing to forget. Do this automatically for the user where possible.
Engage users to think about and clarify what their concepts really mean. Modeling is the process of taking the fuzzy concepts in your head and formalizing them.
Allow users to experiment with new models easily. Keep the cost of rearranging things low.
Allow for multiple ways of looking at (and interacting with) the same model. BCM provides a graphical view that shows all attributes as icons, as well as a natural language summary that goes something like “Each person has a name, has a birthday, may have one or more friends…”
Use visual chunking strategies to group similar concepts together and “treat each group as a single concept.”
A birds-eye view of the system can be disorienting. If a user has to keep the entire system in their head at once, they can lose focus. BCM uses a windows-and-icons approach where each concept has its own window. The user can “pay attention to a single concept at a time” and use the process of placing and navigating between windows as an intuitive way to relate these concepts. I was so surprised to learn that the boxes-and-arrows approach of UML and ER diagrams is more confusing than helpful to most people, but the authors “did not see any evidence that a lack of ‘big picture’ hindered the modelers.” Huh!
The learning curve for BCM was less steep. Not the learning curve for the tool, but the learning curve for think analytically about their model. It makes progress in moving modeling from “hard to learn, hard to master” towards “easy to learn, hard to master.”
A study looking at how kids use natural language to describe the behavior of computer programs (in this case, PacMan).
Kids use rules, events, and constraints to describe the program in a reactive way, without explicit flow control. An event-based style of programming is generally preferred.
Though, “different styles seem to be more natural for different parts of the programming task.” For example, declarative statements are used to set up the scenario.
Rather than iterating over a list one at a time, the kids describe aggregate operations over sets and subsets. They construct sets “using plurals, the keywords each, every, or all, or by naming columns in a table” and then refine those sets using “inverse or difference” or similar operations.
Basic sorting methods should be built-in.
Avoid the need for state variables. Instead of tracking progress with variables, the kids define an end-state for the task. When specific historical information is needed to make decisions, they use “future and past tenses to refer to the needed information.”
An examination of complexity in complex programming projects and how to mitigate it.
Complexity arises from mutable state and hierarchy.
The only state should be the input. Everything else is a view over that input.
Start with a relational database and use relational algebra and pure functions to derive additional state. Use constraints to delimit what changes can be made.
Functional Relational Programming
Out of the Tarpit is about dealing with complexity in programming systems. The authors make a distinction between:
essential complexity: complexity inherent in the problem you’re trying to solve, and
accidental complexity: complexity that arises from implementing a solution.
There’s not a whole lot you can do about essential complexity, so the goal is to limit (and separate out) accidental complexity.
Accidental complexity in programming tends to arise from:
mutable state: allowing side-effects that are hard to track and a proliferation of states that are hard to account for, and
hierarchy: class inheritance being an obvious example that can lead to tightly coupled code that’s hard to refactor.
If that sounds like they’re beating up on your typical object-oriented style programming… that’s because they are. They instead look to functional and logic programming for inspiration. Of course, those languages need to break their own rules for pragmatics’ sake, allowing side-effects and flow control structures to deal with interactivity and performance.
I’d go further and suggest that people tend to think with nouns more than they think with verbs (or, at least English speakers do). Creating a robust programming language does not necessarily lead to one that’s easy to learn or to use. And if you can’t easily reason about a tool, it won’t be used effectively – if it’s used at all.
The proposed solution, then, starts with a relational database as the core data structure. Instead of classes, you have relations. A relation* is a set of records with certain attributes**. Or put another way, a relation is a table with a set of typed, named columns (the attributes) and any number of un-ordered, non-repeating rows (the records).
Your essential state is simply your user input. It might come from some other source, like an API or something, but it contains only the information relevant to the problem. Any other information or state needed for implementing the solution – accidental state – gets derived from these base relations.
Derived relations depend only on base relations. They don’t get modified directly. You can use a combination of relational algebra (joining, intersecting, projecting, selecting, and other fun nonsense***) and pure functions to create them.
Finally you get to input and output.
For input, there are feeders components that convert user input into (base) relations – basically, things that trigger a change in the underlying state.
For output, there are observer components that convert relations (base or derived) into views or other effects.
This is where it starts to look like functional reactive programming. There are well-defined ways in which the system can affect or be affected by the outside world. What’s more, there’s only one place you can change state – the base relations – and everything else grows out from there. Given the same data, you get the same derived relations and views and behavior every time.
There are couple of other parts to this model as well:
There’s the fun addition of constraints, which are simply boolean expressions that must hold true at all times. Every time you try to modify state (changing records in the base relations), every constraint has to pass before anything can change. Basically these are built-in tests.
There are also performance hints, which let you do such things as define a mapping from a relation to its storage format, mark which derived relations should be cached, and define the order and eagerness of evaluation for derived relations.
Overall, this model is dubbed functional-relational programming. The other FRP. OFRP. And so far, the only example I’ve seen of someone trying to implement this is Eve.
You’re hopefully starting to notice some striking similarities between functional-relational programming and AppSheet. We pretty much already treat users’ spreadsheet data as a relational database, and everything in our app model has an analogue in the OFRP model:
Base relations ⇒ tables
Derived relations ⇒ slices
Feeder components ⇒ edit/save/delete buttons in the app
Observer components ⇒ views and workflow rules
What the OFRP model suggests is that we can expand the power of our app model without compromising its simplicity. For example:
Slices could take advantage of more relational algebra functions.
Views could be more modular and closely related to these richer slices to allow for more expressibility (and separate out the data selection/aggregation elements that are currently defined in views).
Users could define reusable functions (which could simplify column and slice formulas).
Constraints could add security and stability (and maybe also simplify column and slice formulas).
We could create richer app interactivity with a wider variety of feeder components.
We could continue to integrate with external tools via observer components.
Internally, we might be able to use performance hints – that’s probably not something we’d want to expose to users unless we have to.
Now, OFRP is not necessarily designed for learnability or ease of use. But in addressing the complexity that makes real-world systems hard to understand and maintain, it suggests that our apps can deal with arbitrarily complex problems without introducing a mess of problems of their own.
Our app model was built with a strongly declarative paradigm, and this has paid off! We’ve converged onto a solution for dealing with program complexity, and even though our apps tackle a small subset of all possible programs, I think we can take a lot of inspiration from functional-relational programming as we look for ways to tighten-up and expand our app model.
* Okay, I’m conflating relations and relvars for the sake of simplicity. “Relvar” is short for “relation variable,” which refers to the structure rather than the contents. In other words, the schema.
** Attributes are typed. They can be any primitive type (string, int, float, date, enum…) but nothing with subsidiary components (tuples, arrays, dictionaries…).
*** Relational algebra is cool and powerful, and there’s a bunch of information out there on making these transformations performative (though it’s sometimes tricky for large data sets).