This is live documentation of my efforts to understand why I woulduse Lisp (any Lisp) in 2021. I understand why it was used extensivelyin previous decades; it had features that no other languages evenclaimed to have, much less support as robustly as Lisp. But I don’tunderstand why I would use Lisp nowadays. Despite this, lotsofpeopleuse Lisp as a day-to-day programming language to get things done. Giventhat I will be responsible for huge chunks of projects for the nextseveral years and not limited by a non-technicalmanager, I think it would behoove me to understand both sides of theargument.1 These are my notes.
Table of Contents
- Why Lisp?
- Motivation
- RelatedWork
- Methodology
- Results
Motivation
I am fully aware (and terrified of) Paul Graham’s Blub Paradox: I don’t knowwhat my programming language of choice (Blub) is missing, so I can’tmiss it. Since reading Graham’s essay, I’ve noticed this happen atleast four times in recent memory:
- Generatorexpressions in Python for lazy evaluation.
- Using
site:<sitename.com>in web searches. - Vim (it’s awesome)
- tmux (also awesome)
- Dash (and I’ve only been usingit for 3 days)
I can’t imagine going back to not using these tools. But I was doing“fine” before learning them. To me, that implies that there are othertools/language features that I could not imagine working without,if I only knew them. Not knowing “can’t-live-without”features seems like I am deliberately hampering my programming ability.In an effort to alleviate that, I want to learn Lisp.
Specifically, I want to enumerate the features in Lisp that I’mmissing and write enough Lisp to understand the value and power of thosespecific features.2
Throughout this document, I’ll refer to these features as Blub++features: features that Blub doesn’t have, but that Blub++would.
Eric Raymond’s How to Become aHacker is widely quoted [emphasis mine]: “Lisp is worth learning fora different reason — the profound enlightenment experience you will havewhen you finally get it. That experience will make you a betterprogrammer for the rest of your days, even if you never actuallyuse Lisp itself a lot.” Greenspun’sTenth Rule, “Any sufficiently complicated C or Fortran programcontains an ad hoc, informally-specified, bug-ridden, slowimplementation of half of Common Lisp,” leads one to believe that mostbig programs end up using Lisp-like features anyways, so why not startwith Lisp? Matthew Butterick tries to alleviate some of these issues inWhyRacket? Why Lisp?. Anurag Mendhekar explainsthat he uses Lisp because it is “an s-expression based, dynamicallytyped, mostly functional, call-by-value lambda-calculus basedlanguage.”
Paul Graham deserves his own section; his advocacy for Lisp isunparalleled in my internet experience. Some particular essays of note:Revenge of the Nerdsmakes a fundamental argument that languages are better and worse thaneach other; that languages can be ranked (in the context of a givenproblem). Graham explains the origins of Lisp and explains that Lisp wasalways designed to be powerful, whereas other languages (Fortran and itsdescendants) were designed to be fast. Both families are converging topowerful and fast nowadays. In 2021, this is an argument bothfor and against Lisp:
- Lisp implementations are sufficiently fast, so Lisp is best.
- Modern languages are powerful, so they are best.
As we’ll see later, there are still Lisp features missing from modernlanguages, so this is probably an arugment for Lisp: you should writeLisp because you are not limited by processing power anymore. Succinctness is Power isan argument that succinctness is…power (great title). I agree thatsuccinctness is a good measure of language power. I don’t agree power isalways the most important goal. Go, Elm and Rust are examples of modernlaguages that do not prioritize power above all else. These languagesare powerful because programmers can implement bug-free code faster (intheory).
ProgrammingBottom-Up is an attractive idea to me because I design programs inthe top-down manner. Designing from the bottom up is another Blub++feature: I don’t know what this is like, and I can’t even imaginehow it would change my development process.
Methodology - WhichFeatures Will I Learn?
Based on my reading, the best features of Lisp are (in no particularorder):
- Everything is an expression
- Emphasis on functional programming
- Dynamic typing
- Macros
- Improved development experience as a result of these aspects
- I become a super-genius hacker-man if I write a lot of Lisp???
Also based on my reading, Lisp seems to be best used in applicationswhere (again, no particular order):
- Developer speed is critical.
- You control the execution environment (you’re not sendingexecutables to clients).
- You are not working on a large team .3
- Performance is not absolutely critical. 4
With those criteria in mind, the other languages that I’m quitefamiliar with that also fit most of these criteria is Python. Python 3(henceforth referred to as just Python) has excellent community support,simple syntax, and a C FFI as aperformance escape-hatch. MyPy also makes writing correct code mucheasier in my opinion.
Other languages that I’m not familiar with thatmight fit these criteria:
- Erlang/Elixir
- Ruby
- Haskell/OCaML (I think? There are advocates for Haskell that thetype system makes development easier/faster.) TODO: source
To evaluate Lisp against Python, I’ll choose features present in Lispthat are not present in Python:
Everything is an Expression
Python does not treat everything as an expression. The classicexample to explain the value of “everything is an expression” ispresented by Butterick in WhyRacket? Why Lisp?, and goes something like this: In Lisp, there areno statements, so any expresssion can be used anywhere. In Python, youcannot use an if-statement as a value in an assignment expression:
x = if foo: 2 else: 4This is quickly refuted by Python’s ternary:
x = 2 if foo else 4Butterick explains that you can do the same thing with operators,which is “impossible” in Python:
((if foo * +) 4 3) ; if foo, then 4 * 3, else 4 + 3Arguably you can do this in Python as well (thanks to the power of immediatelyinvoked function expressions, as I recently learned from Garrett Morse), but it is certainlyless ergnomic:
((lambda x, y: x * y) if True else (lambda x, y: x + y))(4, 3)(It’s almost surprising how much this looks like Lisp.)
It’s clunkier in Python than Lisp, but this example is more possiblein Lisp because it treats * and + as functions, whereas infix operatorsin Python aren’t. If instead of * and + we had multiply andadd, it would be:
((if foo multiply add) 4 3)And:
(multiply if foo else add)(4, 3)So while I agree that not everything is an expression in Python, I’mnot convinced of its utility.
Emphasis on FunctionalProgramming
I think Python has a fairly strong lean towards functionalprogramming. Functions are first-class objects and can be passed aroundas values (as shown above).Listcomprehensions create copies of lists (emphasizing immutability) andcan be used to map and filter lists. Furthermore, map andfilter are builtins that lazily evaluate the effects ofapplyin a function to a list. Reduce is available as functools.reduce,but I definitely use reduce less than map/filter in day-to-dayprogramming. Generatorsallow you to create infinite, lazily-evaluated sequences.
Some missing functional features would be built-in function currying(although it does exist in functools under the name of partial)and enforcing side-effect-free (pure) functions. Other than that, Idon’t know of any functional programming features that I miss in Python.Granted, my only experience with a purely functional language is Elm, so I don’t know howto use monads or typeclasses (although I think typeclasses are Hindley–Milnertype system feature, not a function language feature). There areprobably functional programming features that I’m not even aware of,simply because they aren’t in Python. Given that, I belive thatfunctional programming features available in Lisp that I’m not aware ofmight be worth using Lisp. Once I know of such features, however, I’mnot convinced that I won’t be able to apply that style of thinking toPython.
Dynamic Typing
Python is dynamically typed. I have found dynamic typing lessand less helpful, but I admit that using # type: ignorein some Python code has given me massive flexibility when I need it. ButI don’t think Lisp’s dynamic typing is going to be somehow life-changingcompared to Python.
Macros
This is the big one. A quote from RacketSchool 2019: “You would write such functions [macros] because youwant to abstract over recurring patterns in your code that cannot beabstracted over with functions (or other means of conventionalabstraction).” I understand that Python has some meta-programmingcapapbilities, but I don’t ever use them, besides decorators in thestandard library like functools.lru_cache.
Macros are, as far as I can tell, 100% a good reason to use Lisp.
The Amalgamation of TheseFeatures
ProgrammingBottom-Up seems to suggest that all the features of Lisp cometogether to make development much easier. It’s not any single featurethat makes Lisp amazing, but these features in combination. Chapter 1.5of On Lisp suggeststhat “these new possibilities do not stem from a single magicalingredient.” Perhaps Lisp is only a great language because it has all ofthese features, not any single one.
It is subjective and hard to test, but could be a good reason to useLisp.
Ibecome a super-genius hacker-man if I write a lot of Lisp???
This is obviously a subjective reason to use Lisp. Arguably, writinglots of Python would also make me an excellent programmer, especially ifI were to use the more complex features (like meta-programming) on aregular basis. But my personal experience working in Elm and thenreturning to Python supports the argument that writing code in aradically different language can improve your programming abilityuniversally.
So maybe learning Lisp for the sake of learning is a good reason touse Lisp.
Results - How Did I LearnLisp?
In progress.
When to use Macros
One initial worry I had when learning Lisp was the idea that Iwouldn’t know when to use macros. Whenever I’m introduced to a newabstraction (sum types and pattern matching in Elm, for example), I needto learn when it’s appropriate to use it.5 Butmacros are a whole new class of abstraction; it’s like learning thatfunctions exist. Luckily, Racket School of 2019 has a whole track formacros. Day 3 is all about macros and starts by explaining “a basicframework for when language extension is appropriate.”
A fundamental aspect of language-oriented programming is identifyingthese intended abstractions and the invariants that enforce theirintegrity, then exploiting those invariants to produce a better programthan you would have done without the abstraction.
Arguments Against Lisp
JoelSpolsky: “And I have the ultimate respect for Paul Graham — I thinkthere’s a good probability that in a year or two we will credit him withbeing the man who solved spam. But I think that if you try to ignore thefact that millions of programmers around the world have learned lisp anddon’t prefer to use it, you’re in the land of morbid cognitivedissonance.”
Other Notes
From Revenge of theNerds:
“If you can’t find ten Lisp hackers, then your company is probablybased in the wrong city for developing software.”
Maybe I’m in the wrong city, but I don’t know anyone who writesany Lisp code. We wrote a little bit of Scheme at the end ofCSE 3345, but nobody I know has continued to use Lisp.
[Relevant link] [Source]
Sam Stevens, 2024
