24 Feb 2025
In our session this week, we continued to learn the basics of the Scheme/Racket programming language, working though pages 4-8 of The Little Learner.
As so often happens in close reading, our attention was arrested by an apparently innocuous word: “is.” The word “is” has a peculiar meaning in the language of the book. The authors frequently write that something “is” or “is the same as” something else. For example, they pose the question
What is
?
The answer:
(λ (height)
(* 3.0 height))
Or again later:
The expression
is the same as
There is a curious inversion to their presentation. First they present a series of these examples, where s-expressions are evaluated, some involving closures, in which a higher-order function returns a new function that ‘remembers’ some values passed to the higher-order function. Then, they admit that their use of “is” and “is the same as” is not entirely idiomatic:
This way of remembering arguments passed in for formals of outer functions inside inner functions is known as β-substitution.
In other words, the word “is” actually means “can be transformed into via β-substitution.” Two s-expressions “are the same expression” when they can be transformed in this way. But what does this transformation entail? It entails taking the name of something, e.g. add3, height, area-of-rectangle, and replacing it with its value. Is “is” the right word for this? Is the name of a thing “the same as” the thing?
This use of “is” conflicted with intuitions we had in the group. It seems paradoxical to say that the name of something “is the same as” the thing itself. Of course, it is reasonable in the context of evaluating Scheme code. Whenever the code is run, it will be evaluated, so there is a sense in which the code simply is what it evaluates to. This sense of “is” also makes sense in an intellectual culture dominated by mathematics. In everyday algebra, there is no real distinction between equality and identity.
$$3 + 2$$
really is
$$1 + 4$$
Isn’t it? They are equal. Who cares how they are written? Seeing this “is”, of course, can take some work. How many people can really remember why $$a^2$$ really is $$b^2 + c^2$$ in a right triangle?
In everyday life, we are quite happy to “dereference” or “substitute” names for the things themselves. When I ask you to “pass the pepper,” I’m quite happy when you hand me the pepper grinder. You ask, “Is this what you wanted?” I reply, “Yes, it is!”
But nonetheless there is something alarming in being told that two things “are” one another when you haven’t internalised the substitution process that allows you to move between them. And names do have a reality of which we are sometimes reminded. If I ask a Canadian to “pass me the pepper,” and they give me a capsicum, I may be disappointed.
The whole discussion reminded me of piece by Lewis Carroll, in which a person’s name has a name, which itself has a name, which itself has a name, and so on. I was sure that this infinite regress featured in Gödel, Escher, Bach, but I have tried and failed to find either the Carroll Story or the Hofstadter variation on it! Is an intimation of a thing the same as the name of a thing? Or is the intimation the thing? Or is the name the intimation? Can a vague recollection be substituted for a textual authority? Or only for a vague apprehension…?
We recommence next week on frame 24, at the top of page 9.
12 Feb 2025
Today we commenced The Little Learner, the text that will occupy the group for many months to come. We read the Preface and the first page of Chapter 0.
Our discussion focussed mainly on the book’s authorial persona and implied reader. For those of us in the group who have a mainly adversarial attitude towards AI, the book presented a challenge. Isn’t deep learning interesting and fun? Aren’t the algorithms elegant and surprisingly simple? Shouldn’t everyone dive into this fresh and exciting area of research, and learn how to do it?
To invite the reader into the text, Friedman and Mendhakar carefully establish the reader as a novice, and themselves as kind, avuncular teachers. The reader need only know “high-school maths” and have a minimum of “programming experience.” The book proceeds from these foundations in a strict order, to build up from simple pieces the whole complex machinery of modern deep learning.
As some in the group observed, this “novice” reader was already expected to know some terms of art. Concepts such as “problem domain,” “equalities,” “invariants,” “superset” and “subset” were introduced as though they were the general coinage of the realm. Of course, all textbook writers face the problem that their students need to somehow learn the language that even makes it possible to express knowledge of their subject. How can you learn anything about a topic without having the words to describe the topic? But we as a group are intrigued to see precisely who or what these writers assume an interested and relatively ignorant reader to be as the book progresses.
We discussed the possible ideological implications of the book. Is this a book that subtly asserts a “tech-bro” persona? Or does its goofy and academic tone bespeak a different attitude? In our disciplines, we worry endlessly about surveillance capitalism, about the power of tech billionaires, about the algorithmic mediation of human interaction. The writers of The Little Learner sidestep such issues. Deep leaning is fun. It’s for categorising cat photos, not for empowering intelligence agencies to more rapidly scan citizens’ text messages. It’s something anyone can do as a hobby, rather than a tool used by rich and powerful people to make themselves richer and more powerful.
Everyone agreed the book is fun, and the topic is interesting. We will see in coming months how we can reconcile the fun with the cultural critique.
Any code written in the sessions can be found in the Github repository for this reading.
04 Dec 2024
The end of the affair
At our final meeting for 2024, we completed reading Knuth’s Literate Programming. In the final section, Knuth considers “Retrospects and Prospects” for literate programming. In this section, Knuth is explicit about who literate programming is for: computer scientists and systems programmers, rather than hobbyists. This context justifies many of Knuth’s arguments throughout the essay, about the kinds of literacy assumed by the WEB system. But it also widens the main gap in his philosophy—the gap between the programmer and the reader. He anticipates the programs will become works of literature, which implies a wide readership, but he restricts WEB to a small subset of people, resulting in a restricted writership. In this way Knuth sharpens the literary aspect of his enterprise, for indeed literature too is written by the few for the many to consume.
With that, our first major reading for the group came to an end.
After the end
We spent some time looking at the upshot of Knuth’s new programming system. On his website, he has published many literate programs, in addition to publishing three books written in either WEB or CWEB, which contain the programs for TeX, Metafont and the MMIX virtual machine.
Literate programming has inspired many programming systems, but not in the manner Knuth proposed. He saw WEB as a system for highly skilled programmers to write complex software systems. But WEB (and its descendent CWEB) have not found much use in this domain. Instead, programming language designers have designed ever more capable documentation-generation systems, which allow software to be composed in a more conventional format, but with excellent computer-generated documentation. Python includes pydoc as part of its standard library, for example, while Rust ships with rustdoc. Such tools allow a developer to include documentation in their code, and generate attractive websites for their software. They do not support the creation of elegant books of the kind that Knuth prefers, and in particular, do not free the programmer from the sytax of their chosen programming language.
The Knuthian ideal of literate programming has caught on in a different community: data science. Statisticians, digital humanists, data analysts, lab scientists and others frequently use tools such as EMACS org-mode, RMarkdown and Juypter Notebooks to write their software. This form of literate programming is nonetheless distinct from Knuth’s. Knuth foresaw systems programmers building complex reusable systems using literate tools. Data scientists tend to write more ephemeral, simple programs, which analyse a particular dataset or form the basis for a particular article or report. When a data scientist does take the time to develop a more complex and reusable piece of software, they are more likely to do so in the form of a simple R, Python or Julia package. While it is possible to write such software in a more literate style using tools such as nbdev, this is not a common practice.
It is a pity that Knuth’s vision of programming-as-literature has not gone mainsteam. Source code is the primary medium of communication for millions of people who work every day as programmers. They write software that affects all of us, and if this software were readable by the general public, then the systems that govern our lives could in principle be more open and democratic. Even an experienced programmer can find it difficult to find a reading path through a complex program. If essential pieces of software such as MediaWiki (i.e. Wikipedia), TensorFlow or Bluesky were written and published in a Knuthian style with a linear narrative, then more people might be inclined to read and debate the code.
What next?
We will reconvene in February 2025, as the summer recess draws to a close here in Australia. Stay tuned for our next reading, which will involve the source code of a Large Language Model. Tell then, Happy Holidays!
20 Nov 2024
In this session, we read sections L and M of “Literate Programming,” which consider the economics of WEB and its relationship to prior work.
Economics of a text editor
Where his earlier writing was produced by hand, the result of a careful and meticulous craftsmanship, in the 1950s Faulkner developed a messier, faster method that relied solely on a typewriter. (DiLeonardi 2024, p. 177)
How does a writing tool affect the writer? Knuth argues that WEB code is faster to write, because WEB imposes a salutary discipline on the programmer at the time of composition. It may take a little longer to write Version 0, but the code will be so much better that it will only be a short step to Version 1.
One member of the group drew a comparison with William Faulkner, whose practice changed significantly when he adopted the typewriter in the late 1950s. As DiLeonardi (2024) explains, Faulkner’s typescripts and manuscripts are quite different documents. His handwritten manuscripts are meticulously crafted. Faulkner claimed that he hadn’t changed a word of As I Lay Dying (1930) in the writing, but the flawlessness of the manuscript suggests a close process of real-time editing. His typescripts are different. He complained that it was “too easy” to put words on the page. They are full of typos, and the novels he wrote on a typewriter, such as The Town (1957) are looser and less crystalline that his handwritten books.
Knuth’s adoption of WEB had, in his own estimation, the opposite effect to Faulkner’s adoption of the typewriter. He claims that code he once wrote carelessly he now writes carefully. When he writes a WEB program, he is in “expository mode.” He imagines that he is explaining the code to someone else, and this mode of thought makes for better programming.
Knuth is trying to advocate WEB as a practical programming tool, and couches his arguments in terms an IBM executive might understand: will this tool reduce labour costs? But in our group we observed a different side to the argument. In his discussion of the “expository mode,” Knuth once again undermines his idea about WEB programming as a “stream of consciousness.” The expositor does not simply let their consciousness stream. They shape that stream for the benefit of a student or a reader. Although he doesn’t clearly theorise their role, Knuth is aware of the importance of the reader, and of the rhetorical situation of the programmer.
He does want readers. The section ends with a delightful expression of Knuth’s burning desire to promulgate both his code and his coding tool.
The rhetorical function of acknowledgements
Section M is essentially and “acknowledgements” section, or an “awards acceptance speech,” as two wits of the group put it. What it is doing in the article?
Partly the acknowledgement of prior work justifies his own undertaking. He makes the radical claim that “it is worthwhile to consider every program as a work of literature,” and wants to establish precedents.
Every program? We discussed this with some astonishment. In our own fields—literature and art history—there is habitually some distinction between what is literary or artistic and what is not. A bureaucratic form is certainly writing, but doubtfully literature. Painting a door is certainly painting, but probably not art. Should every program be see as a piece of literature?
There are two problems with Knuth’s claim, one obvious, the other more subtle.
The obvious problem is that many programs are boring, technical or uninteresting. There are programs that resemble shopping lists and bureaucratic forms, as well as programs that resemble haikus or essays. Perhaps Knuth really is advocating a Dadaist aesthetic, and wants to claim that even the humblest little utility program can be dignified as literature, if only it is written and documented properly in WEB.
The more subtle problem is, again, readership. Another class of things we usually exclude from “art” or “literature” are creative works that are intended for coterie audiences. Patents, blueprints, scientific articles—these things are usually excluded from the art-world or the literary sphere. Art is for the “public.” Literature is for the “general reader.” Can literate programs be so? Perhaps Knuth is advocating an esoteric literature. It may appeal to a small audience, but it has the same aesthetic values and careful creative purpose as literature intended for the public.
It seems more likely that every is hyperbole, or simply not a claim that Knuth carefully considered before he made it. But it is tempting to view Knuth as a kind of William Blake of programming, simultaneously furious in his esotericism, unorthodox in his standards of beauty, and utterly committed to his crazy craft as a form of expression that reaches out widely into society and lengthily into the future.
Next week, we conclude our reading of “Literate Programming.” And move on to something new …
References
DiLeonardi, Sean. “Mediation, Stream of Consciousness, and the Faulknerian Voice: As I Lay Dying to The Town.” Twentieth-Century Literature 70, no. 2 (June 1, 2024): 173–98. doi:10.1215/0041462X-11205357.
06 Nov 2024
In this meeting, we read the crucial section of “Literate Programming,” J, in which Knuth lays out his theory of ‘programs as webs’. This section justifies WEB in the most general possible terms: all programs are already webs, Knuth argues. All WEB does is allow the programmer to clearly express the structure that is already there.
Knuth uses the ‘web’ metaphor in an interesting way. Those of us who are used to the Internet and the World Wide Web may think of a ‘web’ as an inherently dynamic or unstable system. We browse or surf the web. We hop from page to page, from app to app, using hyperlinks. We summon up fragments of the web using search engines, or allow recommendation algorithms to summon up fragments for us as we scroll the feeds of our favourite platforms.
This is not what Knuth means by web. For Knuth, a web is a static, well-ordered structure, like the delicately woven web of a spider, or a narrative tapestry whose threads are chronological. A web is for reading from start to finish. A web has a finite set of components, which have been joined carefully by the weaver of the web. Of course, you may use an index to jump to particular joins on the web. You may use cross references to travel along particular strands. But the web itself is single and entire, with a beginning, middle and end.
Structures and structures
A hierarchical structure is present, but the most important thing about a program is its structural relationships. (p. 107)
Knuth argues WEB accomodates both ‘top-down’ and ‘bottom-up’ programming, or rather, it transcends these two approaches. The WEB programmer can start with a top-level description of a program, or they can start by defining subroutines, or they can mix both freely. This freedom to decide between top and bottom at will frees the programmer from the ‘hierarchy’ of the program. Of course, in the end, ‘[a] hierarchical structure is present’: a program must be a single object the computer can execute, comprising smaller parts that lie within it. But the WEB approach allows the programmer to reveal the ‘structural relationships’ of the program: the logical and intellectual links between different parts of the program.
For example, perhaps some global variables are manipulated by subroutine X, and others are manipulated by subroutine Y. From a hierarchical perspective, each global variable and each subroutine is a separate part of the program, on the same level, while the code inside each subroutine is at the next level down, nested within the subroutine. Using WEB, however, the programmer can explicitly reveal the relationships between the variables and the subroutines, for example by declaring the variables next to the subroutines that matter to them, or by building up the subroutines in parts that a clearly related to other global aspects of the program.
There is an interesting slippage in Knuth’s argument. There is the ‘hierarchical structure’ on the one hand, and the ‘structural relationships’ on the other. Both of these are structur(e|al). What makes them different? How are they related?
Knuth implies that there is no single description of a program that is the right one. Programs have many parts, which combine to form the entire program. These parts have many possible relationships: the orderly hierarchy of their execution by the machine is only one set of relationships. The human reader of a program may observe many other sets of relationships in the program that matter to them.
We could think of this in practical terms. A human might use a profiler, observing how and when different parts of the program are called in practice. They might use a flowchart tool to visualise the control flow. They might write out mathematical theorems that characterise the invariants of parts of the program. They might observe the way that the program models the problem domain, the user, the machine itself. There are (possibly) infinitely many ‘structures’ in a program. Knuth’s aim with WEB is to let the programmer structure their program in whatever way will maximise human comprehension of the code.
Psychological correctness vs. [personal] style
Knuth’s theory of coding style is simulataneously aesthetic, cognitive and functional. Code written in WEB should be aesthetically pleasing, according to literary criteria; it should be easily comprehended (or congnised); and it should function correctly.
These three aims don’t always go together, according to Knuth. He gives an example on page 108. Imagine a programmer is writing a function that does a simple data update, but it needs to check the user input for errors. If the programming language obliges the programmer to put the error-checking code first, then they may feel the urge to shrink the error-checking. The error-checking code is tangential to the function: what really matters is the code at the end, which actually performs the data update for which the function is being written. If there are dozens of lines of error-checking code, which make up virtually the whole function, the programmer may find the function aesthetically repulsive. It would be like designing a pencil with a grip so enormous and contorted that you can no longer clearly see the barrel and tip of the pencil itself. In this case, aesthetics pulls against both cognition and functionality. To make the function seem less ugly, the programmer will try to write the error-checking code as concisely as possible, which may mean it is terse and difficult to understand. They will also be tempting to omit error checks, potentially impairing the functionality of the code.
Knuth demonstrates how WEB resolves the contradiction between aesthetics, cognition and functionality. By giving the programmer complete control over the presentation of the code, and the ability to add labels or commentaries to any part of it freely, WEB allows the programmer to achieve any functionality they like without compromising on either aesthetics or cognition.
There is a tight link, and nonetheless a tension between aesthetics and cognition in Knuth’s theory. Knuth argues that the best way to present a program is in the “psychologically correct” order. But he also argues that programmers can and should develop a personal “style” of programming. If there is a “correct” way to present the program, how is there room for individual “style”?
Knuth’s theory of “psychological correctness” is highly individualised. He argues that a program should represent the programmer’s “stream of consciousness” (p. 107)—that is, the program should be written in the order that the programmer conceived of it. He insists throughout the essay that in his own experience, he only ever envisages a program in one order. There is an order in which the program occurred to him, and this is the order in which it must be written. He argues that when he reads another programmer’s code, he can understand their stream of consciousness easily: the program he presents on pages 98-102 of the article is actually no Knuth’s own stream of consciousness, but Edsger Dijkstra’s.
There is a commonsense aspect to this. If the programmer builds up the program logically, then they can communicate this logical process to the reader, who will hopefully find it easer to comprehend what is going on. Knuth does occasionally modify his theory, admitting that the programmer should not simply regurgitate their actual “stream of consciousness,” but shape the program text with the reader in mind.
But Knuth nonetheless presents the idea of “psychological correctness” in such a stark way that its implications are thriling and extreme. Is it true that every program Knuth writes appears to him in exactly one way? Is this a universal experience of programming? We felt in the group that perhaps Knuth is not accounting for his own extreme level of skill and learning—most programmers probably fumble around, and need to experiment, much more than this most famous computer programmer needs to when he writes software. Is it true that we can understand one another’s thought-processes so easily? Many in our group found the presentation of the primes program on pages 98-102 extremely difficult to follow. The program makes many assumptions about the prior knowledge and discursive competence of its readers. Does Knuth believe that there is a single programming literacy that all programmers share, such that the reader of any program can be assumed to be the same kind of person with the same kind of consciousness?
There is something deeply Kantian about Knuth’s views. He seems to believe in a universal rationality, which extends to the task of aesthetic judgment, and which links cognition to the feeling of beauty. As an unreconstructed Romantic, I find this point of view to be very attractive, even if our experience in this very group demonstrates (for the millionth time) that rationality is more contingent and culturally determined that Kantians may like to admit.
We recommence next time partway through section K, on page 108.
