dustycloud/posts/sussman-on-ai.html

title: A conversation with Sussman on AI and asynchronous programming
date: 2015-10-14 14:00
author: Christine Lemmer-Webber
tags: sicp, foss, sussman, scheme, ai, federation, fsf
slug: sussman-on-ai
---
<p class="centered">
  <a href="https://en.wikipedia.org/wiki/Gerald_Jay_Sussman">
    <img src="/etc/images/blog/Jerry_Sussman.jpg" alt="Sussman!" /></a>
</p>

<p>
  A couple weeks ago I made it to the FSF's 30th anniversary party.
  It was a blast in many ways, and a good generator of many fond
  memories, but I won't go into depth of them here.  One particularly
  exciting thing that happened for me though was I got to meet
  <a href="http://groups.csail.mit.edu/mac/users/gjs/">Gerald Sussman</a>
  (of <a href="https://en.wikipedia.org/wiki/Structure_and_Interpretation_of_Computer_Programs">SICP</a>!)
  The conversation has greatly impacted me, and I've been spinning it
  over and over again in my mind over the last few weeks... so I
  wanted to capture as much of it here while I still could.  There are
  things Sussman said that I think are significant... especially in
  the ways he thinks contemporary AI is going about things wrong, and
  a better path forward.  So here's an attempt to write it all down...
  forgive me that I didn't have a real tape recorder, so I've written
  some of this in a conversational style as I remember it, but of
  course these are not the precise things said.  Anyway!
</p>

<p>
  I wasn't sure initially if the person I was looking at was Gerald
  Sussman or not, but then I noticed that he was wearing the same
  "Nerd Pride" labeled pocket protector I had seen him wear in
  <a href="https://www.youtube.com/watch?v=fAY0_pesZ6s">a lecture I had
    watched recently</a>.  When I first introduced myself, I said, are
  you Sussman?  (His first reply was something like to look astonished
  and say, "am I known?")  I explained that I've been reading the <i>Structure and
  Interpretation of Computer Programs</i> and that I'm a big fan of
  his work.  He grinned and said, "Good, I hope you're enjoying it...
  and the jokes!  There's a lot of jokes in there.  Are you reading
  the footnotes?  I spent a lot of time on those footnotes!"  (And this
  point my friend David Thompson joined me, and they both chuckled
  about some joke about object oriented programmers in some footnote I
  either hadn't gotten to or simply hadn't gotten.)
</p>

<p>
  He also started to talk enthusiastically about his other book, the
  <a href="http://mitpress.mit.edu/sites/default/files/titles/content/sicm/book.html">Structure and Interpretation of Classical Mechanics</a>,
  in which classical engineering problems and electrical circuits are
  simply modeled as computer programs.  He expressed something similar
  to what he had said in the forementioned talk, that conventional
  mathematical notation is unhelpful, and that we ought to be able to
  express things more clearly as programs.  I agreed that I find
  conventional mathematical notation unhelpful; when I try to read papers
  there are concepts I easily understand as code but I can't parse the
  symbolic math of.  "There's too much operator overloading", I said,
  "and that makes it hard for me to understand in which way a symbol
  is being used, and papers never seem to clarify."  Sussman replied,
  "And it's not only the operator overloading!  What's that 'x' doing
  there!  That's why we put 'let' in Scheme!"  Do you still get to
  write much code or Scheme these days, I asked?  "Yes, I write
  tens of thousands of lines of Scheme per year!" he replied.
</p>

<p>
  I mentioned that I work on distributed systems and federation,
  and that I had seen that he was working on something that was called
  the <a href="http://groups.csail.mit.edu/mac/users/gjs/propagators/">propagator model</a>,
  which I understood was some way of going about asynchronous
  programming, and maybe was an alternative to the actor model?  "Yes,
  you should read the paper!" Sussman replied.  "Did you read the
  paper?  It's fun!  Or it should be.  If you're not having fun
  reading it, then we wrote it wrong!"  (Here is
  <a href="http://hdl.handle.net/1721.1/44215">the paper</a>,
  as well as the <a href="http://groups.csail.mit.edu/mac/users/gjs/propagators/">documentation/report</a>
  on the <a href="http://groups.csail.mit.edu/mac/users/gjs/propagators/propagator.tar">software</a>...
  see for yourself!)  I explained that I was interested in code that
  can span multiple processes or machines, are there any restrictions
  on that in the propagator model?  "No, why would there be?  Your
  brain, it's just a bunch of hunks of independent grey stuff sending
  signals to each other."
</p>

<p>
  At some point Sussman expressed how he thought AI was on the wrong
  track.  He explained that he thought most AI directions were not
  interesting to him, because they were about building up a solid AI
  foundation, then the AI system runs as a sort of black box.  "I'm
  not interested in that.  I want software that's accountable."
  Accountable?  "Yes, I want something that can express its symbolic
  reasoning.  I want to it to tell me why it did the thing it did,
  what it thought was going to happen, and then what happened
  instead."  He then said something that took me a long time to
  process, and at first I mistook for being very science-fiction'y,
  along the lines of, "If an AI driven car drives off the side of the
  road, I want to know why it did that.  I <i>could</i> take the
  software developer to court, but I would much rather take the AI to
  court."  (I know, that definitely sounds like out-there science
  fiction, bear with me... keeping that frame of mind is useful for
  the rest of this.)
</p>

<p>
  "Oh!  This is very interesting to me, I've been talking with some
  friends about how AI systems and generative software may play in
  with software freedom, and if our traditional methods of considering
  free software still applied in that form," I said.  I mentioned a
  friend of a friend who is working on software that is generated via
  genetic programming, and how he makes claims that eventually that
  you won't be looking at code anymore, that it'll be generating this
  black box of stuff that's running all our computers.
</p>

<p>
  Sussman seemed to disagree with that view of things.  "Software
  freedom is a requirement for the system I'm talking about!"  I liked
  hearing this, but didn't understand fully what he meant... was he
  talking about the foundations on top of which the AI software ran?
</p>

<p>
  Anyway, this all sounded interesting, but it also sounded very
  abstract.  Is there any way this could be made more concrete?  So I
  asked him, if he had a student who was totally convinced by this
  argument, that wanted to start working on this, where would you
  recommend he start his research?  "Read the propagators paper!"
  Sussman said.
</p>

<p>
  OH!  Prior to this moment, I thought we were having two separate
  conversations, one about asynchronous programming, and one about AI
  research.  Suddenly it was clear... Sussman saw these as interlinked,
  and that's what the propagator system is all about!
</p>


<p>
  One of the other people who were then standing in the circle said,
  "Wait a minute, I saw that lecture you gave recently, the one called
  '<a href="https://www.youtube.com/watch?v=O3tVctB_VSU">We Don't
  Really Know How to Compute!</a>', and you talked about the illusion
  of seeing the triangle when there wasn't the triangle" (watch the
  video) "and what do you mean, that you can get to that point, and it
  <i>won't</i> be a black box?  How could it not be a black box?"
</p>

<p>
  "How am I talking to you right now?" Sussman asked.  Sussman seemed
  to be talking about the shared symbolic values being held in the
  conversation, and at this point I started to understand.  "Sure,
  when you're running the program, that whole thing is a black box.
  So is your brain.  But you can explain to me the reasoning of why
  you did something.  At that point, being able to inspect the
  symbolic reasoning of the system is all you have."  And, Sussman
  explained, the propagator model carries its symbolic reasoning along
  with it.
</p>

<p>
  A more contrived relation to this in real life that I've been
  thinking about: if a child knocks over a vase, you might be angry at
  them, and they might have done the wrong thing.  But why did they do
  it?  If a child can explain to you that they knew you were afraid of
  insects, and swung at a fly going by, that can help you debug that
  social circumstance so you and the child can work together towards
  better behavior in the future.
</p>

<p>
  So now, hearing the above, you might start to wonder if everything
  Sussman is talking about means needing a big complicated suite of
  natural language processing tools, etc.  Well, after this
  conversation, I got very interested in the propagator model, and to
  me at least, it's starting to make a lot of sense... or at least
  seems to.  Cells' values are propagated from the results of other
  cells, but they also carry the metadata of how they achieved that
  result.
</p>

<p>
  I recommend that you read the materials yourself if this is starting
  to catch your interest.
  (A repeat: here is <a href="http://hdl.handle.net/1721.1/44215">the paper</a>,
  as well as the <a href="http://groups.csail.mit.edu/mac/users/gjs/propagators/">documentation/report</a>
  on the <a href="http://groups.csail.mit.edu/mac/users/gjs/propagators/propagator.tar">software</a>...
  see for yourself!).  But I will highlight one part that may help
  drive the above points more clearly.
</p>

<p>
  The best way to catch up on this is to
  <a href="https://www.youtube.com/watch?v=O3tVctB_VSU">watch the video
    of Sussman talking about this</a> while
  <a href="http://mcdonnell.mit.edu/sussman_slides.pdf">keeping the slides handy</a>.
  The whole thing is worth watching, but about halfway through he
  starts talking about propagators, and then he gets to an example of
  where you're trying to measure the height of a building by a variety
  of factors, and you have these relationships set up where as
  information is filled in my a cell's dependencies, that cell merges
  what it already knows about the cell with what it just learned.  In
  that way, you might use multiple measurements to "narrow down" the
  information.  Again, watch the video, but the key part that comes
  out of the demonstration is this:
</p>

<pre>
(content fall-time)
=&gt; #(contingent #(interval 3.0255 3.0322)
                (shadow super))
</pre>

<p>
  What's so special about this?  Well, the fall-time has been updated
  to a more narrow interval... but that last part (shadow and super)
  are the symbols of the other cells which propagated the information
  of this updated state.  Pretty cool!  And no fancy natural language
  parsing involved.
</p>

<p>
  There's certainly more to be extrapolated from that, and more to
  explore (the Truth Maintenance Systems are particularly something
  interesting to me).  But here were some interesting takeaways from
  that conversation, things I've been thinking over since:
</p>

<ul>
  <li>
    AI should be "accountable", in the sense that it should be able to
    express its symbolic reasoning, and be held up to whether or not
    its assumptions held up to that.
  </li>
  <li>
    Look more into the propagators model... it's like asynchronous
    programming meets functional programming meets neural nets meets a
    bunch of other interesting AI ideas that have been, like so many
    things, dropped on the floor for the last few decades from the AI
    winter, and which people are only now realizing they should be
    looking at again.
  </li>
  <li>
    On that note, there's so much computing history to look at, so
    many people are focused on looking at what the "new hotness" is in
    web development or deployment or whatever.  But sometimes looking
    backwards can help us better look forwards.  There are ideas in
    SICP that people are acting as if they just discovered today.
    (That said, the early expressions of these ideas are not always the
    best, and so the past should be a source of inspiration, but we
    should be careful not to get stuck there.)
  </li>
  <li>
    Traditional mathematical notation and electrical engineering diagrams
    might not convey clearly their meaning, and maybe we can do better.
    <a href="http://mitpress.mit.edu/sites/default/files/titles/content/sicm/book.html">SICM</a>
    seems to be an entire exploration of this idea.
  </li>
  <li>
    Free software advocates have long known that if you can't inspect
    a system, you're held prisoner by it.  Yet this applies not just
    to the layers that programmers currently code on, but also into
    new and more abstract frontiers.  A black box that you can't ask
    to explain itself is a dangerous and probably poorly operating
    device or system.
  </li>
  <li>
    And for a fluffier conclusion: "If you didn't have fun, we were
    doing it wrong."  There's fun to be had in all these things, and
    don't lose sight of that.
  </li>
</ul>