Nekthuth User Manual

Overview

Nekthuth is simultaneously a plugin for vim and library for Common Lisp that allows developers to interact with a running interpreter from inside their editor.

Getting Started

Requirements

  • vim 7 or later compiled with +python
  • Python 2.3 or later
  • SBCL with threading enabled (tested with 1.0.15)
  • HyperSpec installed for online help

Install nekthuth

Getting the vim component

Download the latest nekthuth and install into your vim plugin directory (for our purposes we will assume that that's ~/.vim). Get the latest version from http://nekthuth.com/releases . 
cd ~/.vim
wget "http://nekthuth.com/releases/nekthuth-vim_latest.tar.gz"
tar zxf nekthuth-vim_latest.tar.gz

Getting the lisp component

Nekthuth is on the cliki so you should be able to simply use (asdf-install:install 'nekthuth) from inside sbcl, however if you would like download it manually, get the latest releases from http://nekthuth.com/releases . The gpg key for me (I use frank@kank.net generically) is at http://kank.net/pubkey.asc . See http://www.cliki.net/ASDF-Install for more information on asdf-install

Hyperspec

Get the hyperspec from LispWorks FTP site and put it somewhere.

Define the *hyperspec-location* to point to that directory in your ~/.nekthuthrc.lisp. 

cat << EOF > ~/.nekthuthrc.lisp
(defvar *hyperspec-location* "/path/to/hyperspec/")
EOF

Test to see that the directory is correct in sbcl (so that nekthuth won't error on you) 

CL-USER> (load (merge-pathnames #p".nekthuthrc.lisp" (user-homedir-pathname)))
T

CL-USER> (probe-file (merge-pathnames #p"HyperSpec/Front/X_AllSym.htm" (pathname *hyperspec-location*)))
#P"/path/to/hyperspec/HyperSpec/Front/X_AllSym.htm"

Setup

Remote execution of lisp code

Import nekthuth into a running session 

CL-USER> (asdf:oos 'asdf:load-op 'nekthuth)
; Redacted comments and warnings
NIL

Start up the nekthuth listener 

CL-USER> (nekthuth:start-remote)
#<SB-THREAD:THREAD "Main Remote Thread" RUNNING {ADE9AD9}>

Local execution of lisp code

Please note that local execution of code is actually more error prone because it has to do all the error checking that you would do manually with remote start (bsd-sockets installed, nekthuth installed, etc.) on boot, so it may be easier to start with the remote setup as listed above.

Setup vim to know where the sbcl executable is (defaults to /usr/bin/sbcl): 

echo 'let g:nekthuth_sbcl = "/path/to/sbcl"' >> ~/.vimrc

Testing your installation

Test your installation by editing a .lisp file and adding a simple line such as (print "Hello world"). To start the interpreter in a second buffer, which should appear on the left, use either the :NekthuthRemote or :NekthuthOpen command, if doing remote execution or local execution, repsectively. Position your cursor over the opening brace and use the :NekthuthSexp command to send the print statement to the nekthuth. If all goes well, your screen should look like the following:

Upgrading to new version

To upgrade, in lisp re-running asdf-install will suffice, while in vim running the above download and tar command will suffice, as I will not remove any files in the foreseeable future.

Basic Features

S-expression evaluation

NekthuthSexp

The basic function of nekthuth is to allow you to send expressions to the interpreter without leaving your vim window. From any open buffer, by positioning your cursor inside some list, you can use the NekthuthSexp command. If you are over an atom that is not inside a list, just the atom will be sent. The intepreter will use the current body of functions and symbols you have defined to executes as if running at the top level. If your s-expression involves some long operation (or a sleep), vim will not be able to immediately print your response. You will see a printline show up in the message area telling you that a response from lisp is waiting. At that time, simply move your cursor to trigger the buffer dump function.

In this example, we'll create a long running command via a sleep statement, and send it through the nekthuth.

After the 5 seconds are up, a message appears at the bottom.

Moving the cursor will let the vim loop print the waiting response.

In this way, even if there is a long running process going, you can still move around vim, or send an interrupt (discussed below).

NekthuthTopSexp

The NekthuthTopSexp command will send the top level S-expression to the interpreter. By using this as well as NekthuthSexp, you can test out individual expressions inside function definitions before sending the whole thing. In the following code sample, positioning your cursor over the first print statement, the let block, and the defun block will result in different output using NekthuthSexp, but will all do the same thing when using NekthuthTopSexp 

(defun say-hello ()
 (let
  ((my-var "world"))
  (print "Hello")
  (print my-var)))

Note that if you position your cursor over "(print my-var)", and use NekthuthSexp, you would get an error as my-var is not bound at the top level.

Assigning shortcut keys, using numeric arguments

Typing the previous commands whenever you want to interpret somthing will become laborious quickly. In my normal setup, I have bound <C-J> to NekthuthSexp and <C-E> to NekthuthTopSexp: 

au FileType lisp map <C-J> :NekthuthSexp<CR>
au FileType lisp map <C-E> :NekthuthTopSexp<CR>

When done in this way, you can preface <C-J> with a number, and it will evaluate that many s-expressions above the one your cursor is in. For instance, in the code sample above, if you were to position your cursor over (code (print "Hello")), and use <C-J>, you would see "Hello" printed to the interpreter buffer. However, if you were to us 2<C-J>, the entire let block would get sent.

Macroexpansion

NekthuthMacroExpand

When your cursor is over an s-expression for which you would like to see the macroexpansion (under macroexpand-1), you can use the :NekthuthMacroExpand command to have vim print the expanded form in the message window below your buffers. In this manner, you can easily see how your code is getting expanded without crowding your interpreter buffer.

Assigning a shortcut key, using numeric arguments

Like NekthuthSexp, when a shortcut key is bound to NekthuthMacroExpand (I bind <C-M>), a number can preface the command to tell nekthuth how many levels up you would like sent to macroexpand-1.

Omnifunction for tab completion

Nekthuth asks the running lisp system for vim style tab completion. Using the omnifunc setting (access with <C-X><C-O>), vim will provide a menu of possible functions/symbols that have currently been defined. You can then select which one you would like to use normally. You can use hyphen syntax such that "m-v-l" can expand to "multiple-value-list" and "multiple-values-limit."

Tab completion supports both functions in the current *package* (and packages that transitively uses), but also external symbols of defined packages in your system. You can use hyphen syntax for those as well, so for instance "s-i:w-r" will expand to both "sb-int:with-rebound-io-syntax" and "sb-introspect:who-references".

Syntax highlighting of defined functions

When the standard lisp syntax file for vim is used, all common lisp symbols will be highlighted (usually in yellow, but it depends on your theme). Nekthuth introduces the lispLocalFunc syntax group, which is aliased to the Function group (usually shows up as Cyan). Whenever a local function/macro is defined in your top level, all occurences of that symbol will be marked as a Function, giving you greated confidence that you have avoided a typo.

This highlighting is extended to the external functions and macros of all packages loaded up on the system. Therefore, if you use asdf to load up something like 'cl-ppcre, then functions such as cl-ppcre:scan-to-strings will be highlighted in blue. For the current *package*, internal functions will also be highlighted as above.

Online vim help

Nekthuth, using the cl-html-parse library, loads up different hyperspec functions much the same way as a text based browser would do inside your vim instance. The tags file distributed with nekthuth points to which webpages each function/symbol refers to. Please note that all the content remains under the Copyright of LispWorks, and should not be distributed in the displayed format (See http://www.lispworks.com/documentation/HyperSpec/Front/Help.htm#Legal for details).

Clhelp

The Clhelp command is used to look up a function in the hyperspec. Tab completion from the tags file is available for Clhelp at the vim command line. You can further use a vim mapping to look up the symbol currently under your cursor: 

au FileType lisp map <C-H> :Clhelp <C-R><C-W><CR>

Debugger

When an error is signaled inside your lisp code, the message window will come up with available restarts, as well as a limited backtrace. You can choose between these, which will get sent to the interpreter such that you can move on. The default, and always available, restart is nekthuth-abort, which will return the REPL to the nekthuth top level.

Known Limitations

The debugger is currently the trickiest part of the entire library. Mainly due to shortcomings in the author, the following may have issues:

  • There is no way to enter a value for restarts such as 'store-value
  • Nekthuth may sometimes freeze if the debugging stacktrace is odd
  • There is no way to expand backtraces or see more of them
  • If you enter a debugger command that causes multiple levels of debugging to happen, nekthuth cannot handle it and you may require a kill from another terminal.
  • There is no option to kill the nekthuth-thread

NekthuthInterrupt

When you enter a long running, or infinite lisp expression, you can use the NekthuthInterrupt command to cause the thread to become interrupted and you to be prompted with a debugger. At that time you can use the normal 'nekthuth-abort restart, or continue on with the 'soldier-on restart. For ease of use, I map this command to <C-C>.

NekthuthSourceLocation

When your cursor is over a symbol, you can use NekthuthSourceLocation to open up the file and location that the function/macro was defined in. If your symbol was defined in the top level through the REPL (or using the load function, rather than compiling and loading the fasl), sbcl cannot collect enough information for Nekthuth to open up the correct file. For ease of use, i map this command to <C-]>.

Remote Connection commands and configuration

When using the following commands, you can enable an already running lisp to start listening for connections from vim instances. If you would like to have two different lisp instances listening, you will need to start the second up with a different port as discussed below. Each listening thread will aaccept multiple connections from however many vim instances you would like to have connected to it, and all of the syntax highlighting should just work between all those instances. Currently localhost is hardcoded into the system.

Starting and stopping the remote connection

To start a thread that listens for remote nekthuth connections inside a running vim, run the following in your lisp instance. Please note that you can only have one of these threads going at once in that running lisp (but why would you want two?). 

CL-USER> (asdf:oos 'asdf:load-op 'nekthuth)
; Redacted comments and warnings
NIL

CL-USER> (nekthuth:start-remote)
#<SB-THREAD:THREAD "Main Remote Thread" RUNNING {ADE9AD9}>

To then stop the thread the thread, use the following. 

CL-USER> (nekthuth:stop-remote)
NIL

Configuration

The default remote port is 8532, and can be used only in only one common lisp session at a time, though many vim windows can connect to it.

Changing the default port

In your ~/.nekthuthrc.lisp, just set the nekthuth variable *remote-port* 

(defvar *remote-port* 31337)

In your vimrc, you need to set it accordingly 

let g:nekthuth_remote_port = 31337
Using a one time port to connect to a different lisp

If you would like to boot up a second lisp to connect to with a vim, you can specify which port to use in the startup functions on both sides. Please note, if you want to connect multiple vims to one lisp, that's supported out of the box and you can just do things as normal. 

CL-USER> (nekthuth:start-remote 31337)
#<SB-THREAD:THREAD "Main Remote Thread" RUNNING {AAE4241}>

And when inside your vim session: 

:NekthuthRemote 31337

Plugins

Nekthuth is extensible by installing plugins. Each plugin is usually comprised of a common lisp part and a vim part. If you are interested in developing a plugin, please see the developer documentation for more information.

Installing plugins

Installing a plugin is a matter of downloading one from the internet (such as my local repository ) and using the installation script that comes with the vim part of the plugin. 
~/.vim/ftplugin/lisp/nekthuth/install.sh randomplugin.nek

Nekthuth home directory

By default the nekthuth home directory where plugins will be installed is in $HOME/.nekthuth/. In that directory two subdirectories will be created: vim/ and lisp/ These are for the vim and lisp components of a plugin respectively. If you would like to change that installation, set your NEKTHUTH_HOME environment variable 

echo "export NEKTHUTH_HOME=/path/to/some/dir/" >> ~/.bashrc

Appendix

Vim Configuration

  • disabling - g:nekthuth_disable
  • splitting - g:nekthuth_vsplit - 0/1 (0 is horizontal, 1 is vertial/default)
  • sbcl location - g:nekthuth_sbcl - "/usr/bin/sbcl" is default
  • remote port - g:nekthuth_remote_port - 8532 is default

Nekthuthrc Configuration

All nekthuth configuration is set using defvar in your ~/.nekthuthrc.lisp. Note that all the variables you set and functinos you define will be executed in the package nekthuth.

  • remote port - *remote-port* - 8532 is default
  • hyperspec location - *hyperspec-location* - nil, and thus disabled, by default

Bash configuration

  • Nekthuth plugin home - NEKTHUTH_HOME - ~/.nekthuth is default

Vim commands (the short list)

  • NekthuthSexp <n> - Send <n>th enclosing form that cursor is part of
  • NekthuthMacroExpand - Send <n>th enclosing form to be macro expanded
  • NekthuthTopSexp - Send the topmost form from current position
  • NekthuthClose - Close the nekthuth, shut down SBCL, close the vim buffer
  • NekthuthOpen - Open up the Nekthuth, start SBCL, and the vim Buffer
  • NekthuthRemote - Connects to a running sbcl that has executed (nekthuth:start-remote)
  • NekthuthInterrupt - Send an interrupt to the REPL
  • NekthuthSourceLocation - Find the source location of the word under the cursor
  • Clhelp <topic> - Query hyperspec for help on <topic>, which is tab completable

My vim mappings

I use the following vim mappings with Nekthuth for my ease of use. 

inoremap <C-L> <C-X><C-O>
imap <Tab> <C-N>

" Can prefix with number
au FileType lisp map <C-J> :NekthuthSexp<CR>
au FileType lisp map <C-E> :NekthuthTopSexp<CR>

" Can prefix with number
au FileType lisp map <C-K> :NekthuthMacroExpand<CR>

" Close and re-open the nekthuth
au FileType lisp map <C-I> :NekthuthClose<CR>:NekthuthOpen<CR>:redraw!<CR>

" Get word underneath cursor
au FileType lisp map <C-H> :Clhelp <C-R><C-W><CR>
au FileType lisp map <C-C> :NekthuthInterrupt<CR>

" Find the source location of the symbol
au FileType lisp map <C-]> :NekthuthSourceLocation<CR>

au BufRead *.asd set ft=lisp

" Dropdown selection colors
highlight TabLine ctermfg=gray
highlight TabLineFill ctermfg=black
highlight TabLineSel ctermfg=white
highlight Pmenu ctermbg=Black
highlight PmenuSel ctermbg=Red
set completeopt=longest,menuone
inoremap <expr> <cr> pumvisible() ? "\<c-y>" : "\<c-g>u\<cr>"
let g:lisp_rainbow = 1

Offficial control characters

  • A - Master control character for startup negotiation
  • C - Find completes
  • D - Send debug command
  • H - Find/parse hyperspec help
  • I - Interrupt the currently running command
  • L - Find Source Location
  • M - Macroexpand
  • Q - Close the instance, or close the thread for in-vim or remote-connect respectively
  • R - Send to the REPL
  • S - Syntax additions