Blogging in Lisp


Lisplog is a templating system that blends Apache and Hunchentoot to aid in the maintenance of a blog-like web site.

It is open source, written in Common Lisp, and the code is at


Submitted by Bill St. Clair on Sat, 03 Oct 2015 17:50:23 GMT is a small collection of simple shell scripts that I've written over the years.

Documented by its README.


Add comment   Edit post   Add post

Chef Dev Newbie

Submitted by Bill St. Clair on Sat, 03 Oct 2015 12:28:48 GMT

I discovered that Chef has an Erlang-based server and is looking for developers. Chef is the king of DevOps for huge installations, though it can manage, for free, smaller installations as well. My introduction was the Chef Style DevOps Kungfu speech video by Chef CTO Adam Jacob at ChefConf 2015. On configuration workstations and target nodes, Chef is largely Ruby scripts, nice simple ones that are amenable to building by people who don't write software for a living. There's a Chef Supermarket filled with Cookbooks full of Recipes submitted by the Chef community. A Supermarket Cookbook is an indexed github archive that you can modify, or use as is, and run on your nodes. A Recipe is a Ruby script. Yeah, the food metaphors are a bit much, but it's light-hearted.


I went through the first two tutorials, learned how to install and run the command line tools on an Ubuntu VM on my Mac and to use them to configure a node on a CloudShare VM that the Chef tutorial spins up for you for free. All straightforward and rock solid. And very well documented. Shiny!

Were I doing DevOps for a company's servers, I would have continued doing tutorials about using Chef. Since I'm a developer considering applying to work for them, I decided I should give a try at building the server. I would guess that most of their customers just use Chef's server, free for small installations or paid for larger ones. But some companies need to run their own. There's a tutorial for installing the server, which looks involved, but not really hard, but I don't just want to run a server, I want to be able to change its code. Chef's code is open source and on their GitHub site. The server source is in the chef-server repository. The Erlang code is in src/oc_erchef.

Being a glutton for punishment, I decided to ignore the system requirements and try the installation on a Scaleway ARM machine. It's hard to beat €3/month for 4 cores, 2 gigs of RAM, and 50 gigs of HD, with lots of bandwidth. I got pretty far into installing the required Ruby gems, when it errored saying it needed an x8664 (to install Sun's Java). Rather than attempt to make it work, I switched over to a $10/month DigitalOcean VM (1 core, 1 gig of RAM, 30 gigs of HD). Still not as much RAM as Chef recommends, but I figured it should at least build. And it mostly did.

After spinning up an Ubuntu 14.04 VM, and adding a swap file, I installed Emacs:

$ sudo apt-get install emacs24-nox emacs24-el info

Then GCC & Erlang dependencies:

$ sudo apt-get install build-essential autoconf git libncurses5-dev
$ sudo apt-get install libssl-dev libssh-dev

And Erlang 17.5:

mkdir -p ~/erlang/release
cd ~/erlang
git clone
ln -s ~/erlang/kerl/kerl ~/bin/
kerl build 17.5 erlang-17.5
kerl install erlang-17.5 ~/erlang/release/17.5
kerl_activate 17.5    # My own simple script

Chef likes at least Ruby 2.0, with Bundler:

sudo apt-get install ruby2.0

# The Ubuntu 14.04 ruby2.0 package installs 1.9.1 and 2.0, with 1.9.1 selected.
# Fix that.
sudo ln -sf ruby2.0 /usr/bin/ruby
sudo ln -sf gem2.0 /usr/bin/gem

sudo apt-get install ruby2.0-dev bundler

Ready to clone the server code:

mkdir -p ~/chef
cd ~/chef
git clone

The installation appears to be intended to be done by root, but I decided to do it from a regular account. It worked fine, but I had to change a few normally root-only directories to world-writeable. I suppose I could have just changed them to be owned by my account. Maybe I would have seen fewer complaints about them in the logs.

sudo chmod 777 /var/lib/gems/2.0.0
sudo mkdir -p /var/cache/omnibus
sudo chmod 777 /var/cache/omnibus
sudo mkdir -p /opt/opscode
sudo chmod 777 /opt/opscode

There's a whole bunch of Ruby code to download and build:

cd chef-server/omnibus
bundle install --binstubs

I watched it happen from Emacs, showing the build shell in one pane and dired in the other, sorted by time descending.

$ ls -w 90 /var/lib/gems/2.0.0/gems
addressable-2.3.8             hitimes-1.2.2                rake-10.4.2
berkshelf-3.3.0               httpclient-           retryable-2.0.2
berkshelf-api-client-1.3.0    ipaddress-0.8.0              ridley-4.2.0
buff-config-1.0.1             json-1.8.3                   rspec-3.3.0
buff-extensions-1.0.0         libyajl2-1.2.0               rspec-core-3.3.2
buff-ignore-1.1.1             method_source-0.8.2          rspec-expectations-3.3.1
buff-ruby_engine-0.1.0        mime-types-2.6.1             rspec-its-1.2.0
buff-shell_out-0.2.0          minitar-0.5.4                rspec_junit_formatter-0.2.3
builder-3.2.2                 mixlib-authentication-1.3.0  rspec-mocks-3.3.2
celluloid-0.16.0              mixlib-cli-1.5.0             rspec-support-3.3.0
celluloid-io-0.16.2           mixlib-config-2.2.1          ruby-progressbar-1.7.5
chef-12.4.1                   mixlib-log-1.6.0             sawyer-0.6.0
chef-config-12.4.1            mixlib-shellout-2.1.0        semverse-1.2.1
chefspec-4.3.0                mixlib-versioning-1.1.0      serverspec-2.21.0
chef-sugar-3.1.1              multi_json-1.11.2            sfl-2.2
chef-zero-4.2.3               multipart-post-2.0.0         slop-3.6.0
cleanroom-1.0.0               net-http-persistent-2.9.4    solve-1.2.1
coderay-1.1.0                 net-scp-1.2.1                specinfra-2.41.1
dep_selector-1.0.3            net-ssh-2.9.2                syslog-logger-1.6.8
dep-selector-libgecode-1.0.2  net-ssh-gateway-1.2.0        systemu-2.6.5
diff-lcs-1.2.5                net-ssh-multi-1.2.1          thor-0.19.1
erubis-2.7.0                  net-telnet-0.1.1             timers-4.0.1
faraday-0.9.1                 nio4r-1.1.1                  uber-s3-0.1.1
fauxhai-2.3.0                 octokit-3.8.0                uuidtools-2.1.5
ffi-1.9.10                    ohai-8.5.1                   varia_model-0.4.0
ffi-yajl-2.2.2                plist-3.1.0                  wmi-lite-1.0.0
hashie-2.1.2                  pry-0.10.1
highline-1.7.3                rack-1.6.4

Buildig the server requires your git identity:

git config --global ""
git config --global "Your Name"

Finally I was ready to build the server proper:

cd ~/chef/chef-server/omnibus
bin/omnibus build chef-server

This took a very long time. Reminiscent of Gentoo Linux builds (which took overnight to build Open Office many years ago).

$ ls -w 90 /opt/opscode/embedded/bin
2to3            find2perl                pg_basebackup     resty
a2p             fz                       pgbench           ri
appbundler      gem                      pg_config         rspec
berks           generate_field_data      pg_controldata    ruby
bundle          genhash                  pg_ctl            run_erl
bundler         h2ph                     pg_dump           runit
bunzip2         h2xs                     pg_dumpall        runit-init
bzcat           htmldiff                 pg_receivexlog    runsv
bzcmp           httpclient               pg_resetxlog      runsvchdir
bzdiff          idle                     pg_restore        runsvdir
bzegrep         infocmp                  pg_standby        runsvdir-start
bzfgrep         infotocap                pg_test_fsync     s2p
bzgrep          initdb                   pg_test_timing    sequel
bzip2           instmodsh                pg_upgrade        serverspec-init
bzip2recover    irb                      piconv            shasum
bzless          json_pp                  pkg-config        shell-quote
bzmore          knife                    pl2pm   
c2ph            ldiff                    pod2html          splain
captoinfo       libnetcfg                pod2latex         sqitch
chef-apply      libtool                  pod2man           sv
chef-client     libtoolize               pod2text          svlogd
chef-shell      lzcat                    pod2usage         tabs
chef-solo       lzcmp                    podchecker        testgdbm
chef-zero       lzdiff                   podselect         testrb
chpst           lzegrep                  postgres          thor
clear           lzfgrep                  postmaster        tic
clusterdb       lzgrep                   prove             toe
coderay         lzless                   pry               to_erl
config_data     lzma                     psed              tput
corelist        lzmadec                  psql              tset
cpan            lzmainfo                 pstruct           typer
cpan2dist       lzmore                   ptar              unlzma
cpanm           makedepend               ptardiff          unxz
cpanp           minitar                  ptargrep          utmpset
cpanp-run-perl  moose-outdated           pydoc             uuid
createdb        mzn-gecode               python            uuid-config
createlang      ncurses5-config          python2           vacuumdb
createuser      ncursesw5-config         python2.7         vacuumlo
c_rehash        node                     python2.7-config  xml2-config
ct_run          nokogiri                 python2-config    xmlcatalog
dbilogstrip     npm                      python-config     xmllint
dbiprof         ohai                     rabbitmqctl       xslt-config
dbiproxy        oid2name                 rabbitmq-env      xsltproc
dialyzer        omnibus-ctl              rabbitmq-server   xsubpp
dropdb          openssl                  rackup            xz
droplang        package-stash-conflicts  rake              xzcat
dropuser        pcre-config              rdoc              xzcmp
ecpg            pcregrep                 rebar             xzdec
enc2xs          pcretest                 redis-benchmark   xzdiff
epmd            perl                     redis-check-aof   xzegrep
erb             perl5.18.1               redis-check-dump  xzfgrep
erl             perlbug                  redis-cli         xzgrep
erlc            perldoc                  redis-sentinel    xzless
erubis          perlivp                  redis-server      xzmore
escript         perlthanks               reindexdb         zipdetails
ffi-yajl-bench  pg_archivecleanup        reset

It finally started compiling the Erlang code, which it retried three times, with increasingly long pauses in between

[Builder: oc_erchef] W | [1/3] Failed to execute command. Retrying in 10 seconds...
[Builder: oc_erchef] W | [2/3] Failed to execute command. Retrying in 20 seconds...
[Builder: oc_erchef] W | [3/3] Failed to execute command. Retrying in 40 seconds...
===> Compiling oc_chef_authz                                                                                                                                  
===> Compiling .../oc_erchef/.../oc_chef_authz/src/oc_chef_policy.erl failed                                                
.../oc_erchef/.../oc_chef_authz/src/oc_chef_policy.erl:26: behaviour chef_object undefined

Apparently, chef-server/src/oc_erchef/apps/oc_chef_authz needs to be built in an environment with the chef-server/src/oc_erchef/apps/chef_objects/src/chef_object.erl module available, to provide the chef_object behaviour. I may figure this out sometime soon, but I won't be working much this weekend.

2 comments   Edit post   Add post

Kerl Rocks!!

Submitted by Bill St. Clair on Fri, 02 Oct 2015 10:41:59 GMT

I interviewed with Basho last week, hoping for a job writing Erlang for their Riak distributed database. They didn't hire me, at least not yet, but their system is open source, so I decided one way to convince them I'm capable, and to ground the Erlang study I've been doing for the last couple of months, is to submit some pull requests. They ship binaries built against their fork of version R16B03 of Erlang, and Riak builds in 17.5, but you get build errors for 18.x, so I decided to fix those build errors.


The build errors are mostly due to erlang:now() being deprecated and most of the Riak source set to error on warnings, so the fixes are easy. You either change to os:now(), which returns the same form of result, but doesn't guarantee a unique value each time, or you call erlang:timestamp(), falling back to erlang:now() if it isn't defined (which it isn't before 18.x). time_compat:timestamp() from time_compat.erl does exactly that. Easy fixes, but testing needed in the Basho versions of Erlang 16, 17, and 18 before a pull request is ready for submission.


And that's where Yurii Rashkovskii's kerl comes in. It makes it very easy to download source, build, install, and choose from a bunch of different versions of Erlang. Now would be a good time to click Yurii's photo below and read the README for kerl. I'll go over some of the details, but only what I've used so far.

Yurii Rashkovskii
Yurii Rashkovskii

On a new Ubuntu 14.04 machine, I had to install some packages to get Erlang to build:

$ sudo apt-get install build-essential git libncurses5-dev
# not strictly necessary, but enables those Erlang applications
$ sudo apt-get install libssl-dev libssh-dev

You'll also need to install Sun's Java SDK, if you need the jinterface application and java_src from the ic application (necessary, or at least desireable for Riak).

kerl, like, and rebar is a single file, containing a shell script (rebar's script is mostly Erlang binary). Here's how I installed it on my machine (~/bin is in my PATH):

$ mkdir -p erlang
$ cd ~/erlang
$ git clone
$ ln -s ~/erlang/kerl/kerl ~/bin/

And here's how I know it worked:

$ kerl
kerl: build and install Erlang/OTP
usage: .../bin/kerl <command> [options ...]

         Command to be executed

Valid commands are:
  build    Build specified release or git repository
  install  Install the specified release at the given location

To find which releases kerl knows about:

$ kerl list
usage: .../bin/kerl list 
$ kerl list releases
R10B-0 ... R16B03 ... 17.4 17.5 18.0 18.1
Run ".../bin/kerl update releases" to update this list from

Download and install one of them:

$ kerl build R16B03 erlang-r16b03
Downloading otp_src_R16B03.tar.gz to .../.kerl/archives
Verifying archive checksum...
Checksum verified (c330150913556a0fe73e57a441cb6375)
Extracting source code
Building Erlang/OTP R16B03 (erlang-r16b03), please wait...
Erlang/OTP R16B03 (erlang-r16b03) has been successfully built

kerl doesn't print out much while building. To see progress, you can look at the log file (from another shell window, the log file is deleted when the build completes successfully):

$ cd ~/.kerl/builds/erlang-r16b03/
$ tail -f otp_build_R16B03.log

The Erlang configure script omits applications for which you have no library support on your system. Here's how to find which ones the build will skip:

$ cd ~/.kerl/builds/erlang-r16b03/
$ find . -name SKIP

Install the build you just made in a directory of your choosing:

$ mkdir -p ~/erlang/release
$ kerl install erlang-r16b03 ~/erlang/release/r16b03
Installing Erlang/OTP R16B03 (erlang-r16b03) in .../erlang/release/r16b03...
You can activate this installation running the following command:
. .../erlang/release/activate
Later on, you can leave the installation typing:

Start using the new installation:

$ kerl list installations
erlang-17.5 .../erlang/release/17.5
erlang-18.1 .../erlang/release/18.1
erlang-r16b03 .../erlang/release/r16b03
$ KERL_ENABLE_PROMPT=y . ~/erlang/release/r16b03/activate
(erlang-r16b03)$ which erl
(erlang-r16b03)$ erl
Eshell V5.10.4  (abort with ^G)
(erlang-r16b03)$ kerl_deactivate
$ which erl

I rarely use kerl_deactivate. Instead I just activate a different version.

Riak builds require Basho's special patched Erlang versions. You can use kerl for them, too. Below I build the basho-otp-16 branch of Basho's OTP fork:

$ kerl build git
usage: .../bin/kerl build git <git_url> <git_version> <build_name>
$ kerl build git basho-otp-16 basho-r16
Checking Erlang/OTP git repository from
Building Erlang/OTP basho-r16 from git, please wait...
Erlang/OTP basho-r16 from git has been successfully built
$ kerl install basho-r16 ~/erlang/release/basho-r16
Installing Erlang/OTP git (basho-r16) in .../erlang/release/basho-r16...
$ KERL_ENABLE_PROMPT=y . ~/erlang/release/basho-r16/activate
(basho-r16)billstclair@Gabriel:~/erlang/release$ erl
Eshell V5.10.3  (abort with ^G)
1> halt().
(basho-r16)$ kerl_deactivate

KERL_ENABLE_PROMPT=y is responsible for the (basho-r16) prefix on the shell prompt. If you don't like that, don't use it. I have a shell script that automates most of the activation. I'll document it soon, along with some of my other useful scripts, with code on GitHub.

$ . kerl_activate basho-r16
(basho-r16)$ kerl_deactivate

kerl makes it easy to test my Riak patches in the three most recent Erlang versions. It has more features, but you already clicked on Yurii's photo, right? So you know about them.

Add comment   Edit post   Add post

SSL Certificate Added

Submitted by Bill St. Clair on Fri, 02 Oct 2015 09:10:46 GMT

I got an SSL certificate from I use them because they're free and recognized by major browsers. I'm looking forward to the advent of Let's Encrypt, which is due in Q4 of 2015. Finally, free, automated SSL certificates, recognized by all major browsers.

Strangely, StartSSL denied my initial request for a certificate. Why? I had a donation button prominently visible at the top of the right-hand column. So I removed it, re-applied, and they approved. Go figure. I hadn't gotten any donations anyway. But if you're so moved, PayPal to the email at the bottom of the page.

The bottom line is that you can now reach this site via

Add comment   Edit post   Add post

Mac OS X El Capitan Startup Transients

Submitted by Bill St. Clair on Thu, 01 Oct 2015 15:46:56 GMT

If you have a Mac, and lots of lispers do, you probably know that yesterday Apple released Mac OS X El Capitan, version 10.11. I'm usually more eager than is healthy to jump on new releases. Not so eager that I feel I have to get the developer releases, or even the public betas, but when a golden master comes out the door, for my Mac or my iPhone, I want it.

But I know to be at least a little cautious. So I followed MacWorld's advice about How to Make a Bootable OS X 10.0 El Capitan Installer Drive, and re-partitioned one of my mostly-empty external drives into three, the existing stuff, a partition to install El Capitan, and a tiny 10 gigabyte partition for the installer itself.

cd /Applications/Install\ OS\ X\ El\
sudo ./createinstallmedia --volume /Volumes/installer \
     --applicationpath /Applications/Install\ OS\ X\ El\

I tried to set the new install partition as my startup drive in System Preferences, but it wasn't a choice. So I shut down, restarted, and held down the <option> key. It WAS one of the choices there. My external drive is a slow USB-2 drive, so I didn't expect lighting, but it said it would take 8 minutes. Yow! Well, that 8 minutes was actually 15, the last 7 of which at it saying "About a second left". Then it took another 24 minutes, pretty accurately predicted, to finish the install.

I moved Emacs, XCode, CCL, and Erlang, from my 10.10 drive to 10.11 and downloaded Karabiner from the web (so that I could use the <fn> key on my Apple bluetooth keyboard as <control>). I ran XCode, so it would do its initialization stuff. Emacs & Erlang just worked. CCL started, and recompiled its lisp code, but failed at rebuilding its kernel. Something about a missing include file. I reinstalled the command-line tools, and it worked.

xcode-select --install

At this point I'd had enough. I could have tried other things, but I decided to install on my main system. That took just a few minutes to complete the setup, then an hour and a quarter after the reboot to do the upgrade, estimated at 31 minutes, which rose quickly to 37 and fell slowly. I really wish Apple's installer gave a little more feedback about what it's doing. It sometimes seems to be stuck. WIth the old installer, you could open a progress window and see somewhat frightening messages, but at least some progress.

The new system came up. I had to re-enter some passwords for Mail, and I had to reinstall the command-line tools, but nearly everything I use works: Emacs, CCL, Erlang (and rebuilding Erlang with kerl), Mailplane, Firefox, VMWare Fusion (6.0.6), X11 & Gnumeric, Skype, Ignite (midi keyboard player), Adium (client for IRC, Jabber, and just about every other known chat system), Letterspace (notes app that syncs with iPhone), Nimbus (client for and of Course Apple's own Messages, Safari, Contacts, and Calendars.

A couple of things that didn't work were MenuMeters, an updated version of which is available here, and Audio Hijack Pro, which I haven't yet tried to fix.

Also Mail now organizes conversations by actual conversations, not just by matching subject lines. I depended on the former behavior for my Fail2Ban notifications on a web site I adminster. It let me easily tell if a particular IP was causing enough problems to ban it permanently instead of just Fail2Ban's ten minutes at a time. I requested the old behavior as a preference here.

So what's new? Nothing much that I've noticed so far. Maybe it's a little snappier.

1 comment   Edit post   Add post

Ready... Set... Blog...

Submitted by Bill St. Clair on Thu, 01 Oct 2015 14:49:28 GMT

I haven't posted much to this site in a long time. I envisioned it as a place to document and announce new features for my Lisp blogging software: Lisplog. But I haven't made any changes to it in a long time, and I never did the promised documentation. It still needs to automate the setup of the settings and .htaccess files. No telling when or even if I'll get that done. Nobody has been beating down my door for it.

So, I'm going to change the way I think about this site. I'll now use it for blogging about Lisp and functional languages in general, including a lot about my new interest: Erlang. I already have at least three posts in mind: "OS X El Capitan Startup Transients", "Useful Shell Scripts", and "Kerl: Erlang Source Builds and Configuration Made Easy".

Watch this space. note the "XML" link on the top of the right column, and read the "news" feed, which already aggregates some lisp-oriented sites, and to which I'll soon be adding Erlang-oriented sites. If you have recommendations for sites to add to the list, either comment on this post, or send email to the address at the bottom of the page.

1 comment   Edit post   Add post

Seeking Work

Submitted by Bill St. Clair on Mon, 31 Aug 2015 14:21:50 GMT

I have been laid off from my dream telecommuting job with Clozure Associates, so I'm looking for work.

My resumé is here.

2 comments   Edit post   Add post

Feeds List for RSS Aggregator

Submitted by Bill St. Clair on Sat, 04 May 2013 15:03:52 GMT

Today I went live with a new feature for the RSS aggregator, a feed list. It's accessible via the feeds link on the right side of the news index page (in the "Navigation" section of the right column). I'll add OPML and a lisp list of feed URLs soon.

Add comment   Edit post   Add post

Lisp News Aggregator

Submitted by Bill St. Clair on Sat, 06 Apr 2013 14:23:32 GMT

I compiled a list of a few Lisp-related RSS feeds to aggregate, and pointed the new code at them. Accessible via the "news" link in the "Navigation" section of the right column, which links to Here's what I found in a quick search:

I'll add more as I come across them. Comment on this post to recommend your favorites.

Add comment   Edit post   Add post

RSS Aggregator Goes Live

Submitted by Bill St. Clair on Sat, 06 Apr 2013 07:35:01 GMT

After running it on my iMac for a couple of nights, I made Lisplog's new RSS aggregator live at . There are still lots of features I could add, but it's now functional. I'll put together a Lisp blog aggregator here real soon now.

Add comment   Edit post   Add post