Never walk alone...

git tips and tricks - Part 1: the fundamentals

gajon@gajon.org (Jorge Gajon) — Sat, 11 Jun 2022 00:00:00 +0000

Most of the time we only use a very limited set of git commands, git checkout -b <branch> to start a new branch, git pull origin to update your current branch with any remote changes, git add . & git commit -m to save your work, and git push origin to push your changes to a remote repository.

That is perfectly fine, and online tools like Github or Gitlab make working with a git repository even easier. But have you wondered what is going on when we execute those commands?

In this and the following articles, we will explore how a git repository works, and how when you use the commands we mentioned above the branches and commits are created and tracked inside the repository. We will also understand how to fix common problems that arise when working with other people over the same repository.

All of this will be explained in a very easy-to-understand manner, without diving into deep technical details that are not important to our usage of git.

What is a git repository?

The following explanation is not 100% technically accurate, but instead, a simplification that will allow us to think about this topic without getting too distracted with details we don't need.

We can think of a git repository as a collection of commits organized in a Directed Acyclic Graph. A what? A Directed Acyclic Graph (DAG) is a collection of nodes with connections between them, these connections have a direction, and it is impossible to form a closed loop by following these connections.

The following is an example taken from Wikipedia. Notice that no matter which node you pick, there is no way to follow the arrows and end up in the same starting node.

You can think of the git commands as commands to manipulate this DAG. Each node in the graph represents a commit and each commit has one or more parents. In turn, each commit represents a change (or patch) to the contents of your repository.

Branches are pointers to commits

We can refer to the commits in a graph by their hash, each node in a git repository has a unique hash, and no two commits can ever have the same one. But using hashes would be very inconvenient, so we use branches to refer to commits instead.

Let’s see this in action with an example. Let’s initialize an empty repository and create a couple of commits.

$ git init --initial-branch=main test-repository
Initialized empty Git repository in /Users/foo/test-repository/.git/

$ cd test-repository/
$ echo "This is our first file" > README.txt
$ git add README.txt

$ git commit -m "Initial commit, added README.txt"
[main (root-commit) c8c57bf] Initial commit, added README.txt
 1 file changed, 1 insertion(+)
 create mode 100644 README.txt

We created an empty repository with the name test-repository, this created a directory with that name, and we also specified the name of our initial branch to be main.

We created a file README.txt with one line of text, and we committed our work with our very first commit. Notice that git is telling us what the hash of this commit is with [main (root-commit) c8c57bf], in this case, the hash is c8c57bf.

A visual representation of our graph looks like this:

┌────┐    ┌────┐     .─────────.
│HEAD│───▶│main│───▶(  c8c57bf  )
└────┘    └────┘     `─────────'

NOTE: If you follow these commands on your computer you’ll notice that the commit hashes will be entirely different for you, this is normal.

So far we have only one node with commit hash c8c57bf, and a branch named main pointing to this commit. There’s also a special pointer named HEAD that is pointing to main.

Now let’s create a second commit.

$ echo "Welcome to our new repository" >> README.txt
$ git add README.txt

$ git commit -m "Added second line to README.txt"
[main 3bbdb69] Added second line to README.txt
 1 file changed, 1 insertion(+)

The hash of the second commit is 3bbdb69. Our graph now contains two nodes and it looks like this:

┌────┐    ┌────┐     .─────────.
│HEAD│───▶│main│───▶(  3bbdb69  )
└────┘    └────┘     `─────────'
                          │
                          │
                          ▼
                     .─────────.
                    (  c8c57bf  )
                     `─────────'

The newest commit 3bbdb69 has an arrow pointing to the first commit c8c57bf. We say that the parent commit of 3bbdb69 is c8c57bf. The very first commit in a repository does not have a parent, and every subsequent commit will have one or more parents.

Also notice that the branch main has moved and it is pointing to 3bbdb69 and that the HEAD pointer also moved with main.

Let’s add a third commit to make this more interesting.

$ touch empty-file.txt
$ git add empty-file.txt
$ git commit -m "Added an empty file"
[main bece83c] Added an empty file
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 empty-file.txt

Our graph now looks like this:

┌────┐    ┌────┐     .─────────.
│HEAD│───▶│main│───▶(  bece83c  )
└────┘    └────┘     `─────────'
                          │
                          │
                          ▼
                     .─────────.
                    (  3bbdb69  )
                     `─────────'
                          │
                          │
                          ▼
                     .─────────.
                    (  c8c57bf  )
                     `─────────'

The special HEAD pointer will follow us wherever we jump in the graph of commits. By using git checkout we can jump to any commit we want, and it will also take care of updating our working files to match the state they had when that commit was created.

$ git checkout c8c57bf
Note: switching to 'c8c57bf'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at c8c57bf Initial commit, added README.txt

git is telling us that we are now in a detached HEAD state. This means that HEAD is no longer pointing to a branch. This is important because if you create new commits while in this detached HEAD state and then jump back to a different branch, you could lose those commits.

The new recommended way to switch to a detached HEAD state is to use the command git switch --detach, this way you are being more explicit about your intention. For example:
$ git switch --detach c8c57bf
HEAD is now at c8c57bf Initial commit, added README.txt

We can see that the contents of our files and working directory reflect the state they had when the commit we are pointing to now was created.

$ cat README.txt
This is our first file

$ cat empty-file.txt
cat: empty-file.txt: No such file or directory

We can experiment while in this detached HEAD state. For example, we can create new commits and they won’t affect other branches. If we want to discard this experiment we can simply jump to another branch. If we want to keep the changes made in this experiment then we have the option of creating a new branch here.

Open your favorite code editor and change the README.txt file so that it contains the following lines:

This is our first file

Changelog:
- Added README.txt

Then save that change in a new commit.

$ git add README.txt
$ git commit -m "Added changelog to README.txt"
[detached HEAD fb75aff] Added changelog to README.txt
 1 file changed, 3 insertions(+)

Our graph now looks like the following. Notice that HEAD moved to point to the new commit fb75aff, and the new commit has an arrow to c8c57bf meaning that c8c57bf is the parent of our new commit. We are still in a detached HEAD state.

                 ┌────┐     .─────────.
                 │main│───▶(  bece83c  )
                 └────┘     `─────────'
                                 │
                                 │
                                 ▼
┌────┐     .─────────.      .─────────.
│HEAD│───▶(  fb75aff  )    (  3bbdb69  )
└────┘     `─────────'      `─────────'
                │                │
                └───────────────┐│
                                ▼▼
                            .─────────.
                           (  c8c57bf  )
                            `─────────'

Now let’s create a new branch that will point to this new commit, and in this way, we avoid losing this work if we jump to a different branch.

$ git switch -c changelog
Switched to a new branch 'changelog'

Notice that now HEAD points to our new branch changelog, we are no longer in a detached HEAD state.

                 ┌────┐     .─────────.
                 │main│───▶(  bece83c  )
                 └────┘     `─────────'
                                 │
                                 │
┌────┐  ┌─────────┐              │
│HEAD│─▶│changelog│              │
└────┘  └─────────┘              │
          │                      │
          │                      ▼
          │  .─────────.    .─────────.
          └▶(  fb75aff  )  (  3bbdb69  )
             `─────────'    `─────────'
                  │              │
                  └─────────────┐│
                                ▼▼
                            .─────────.
                           (  c8c57bf  )
                            `─────────'

If we create new commits now, the new branch changelog will move to point to the new commits, and the HEAD pointer will move as well to continue pointing to our new branch.

Going back to our main branch

Let’s go back to our main branch, this will make our special HEAD pointer move to point to main and this will also update the contents of our working files to match the state they had when we last created a commit in main.

$ git switch main
Switched to branch 'main'

Our graph now looks like this:

┌────┐       ┌────┐     .─────────.
│HEAD│──────▶│main│───▶(  bece83c  )
└────┘       └────┘     `─────────'
                             │
                             │
    ┌─────────┐              │
    │changelog│              │
    └─────────┘              │
      │                      │
      │                      ▼
      │  .─────────.    .─────────.
      └▶(  fb75aff  )  (  3bbdb69  )
         `─────────'    `─────────'
              │              │
              └─────────────┐│
                            ▼▼
                        .─────────.
                       (  c8c57bf  )
                        `─────────'

And if we inspect the contents of our README.txt file we see that the changes made in the changelog branch are not there anymore.

$ cat README.txt
This is our first file
Welcome to our new repository

Display the repository graph with `git log`

I have shown you a simplified visual representation of the graph that contains our commits inside our repository. You can ask git to display this graph in your terminal with the following command.

$ git log --graph --all
* commit fb75affebd2d44fd7100bfe932a06004623825cf (changelog)
| Author: Jorge Gajon <gajon@gajon.org>
| Date:   Sun Jun 26 23:56:14 2022 -0500
|
|     Added changelog to README.txt
|
| * commit bece83c61b9d0bd58f5d7d90cb9ca10af8771802 (HEAD -> main)
| | Author: Jorge Gajon <gajon@gajon.org>
| | Date:   Sun Jun 26 23:42:06 2022 -0500
| |
| |     Added an empty file
| |
| * commit 3bbdb697b2fe653de8f73c9071bcacb3aaa3b79b
|/  Author: Jorge Gajon <gajon@gajon.org>
|   Date:   Sun Jun 26 23:35:26 2022 -0500
|
|       Added second line to README.txt
|
* commit c8c57bfe49e8d5d1918571f4cbdc708cfd82501e
  Author: Jorge Gajon <gajon@gajon.org>
  Date:   Sun Jun 26 23:15:13 2022 -0500

      Initial commit, added README.txt

Notice a few things. First, the commit hashes are being displayed in their long-form, rather than the short form we’ve seen so far. For example the commit hash we’ve previously seen as bece83c is being shown as its full value of bece83c61b9d0bd58f5d7d90cb9ca10af8771802. Both the short and long-form are interchangeable.

The second thing to notice is that the commit being at the very top is the latest commit we created, in this case, the latest commit in the changelog branch.

If we re-arrange a little bit the visual graph we were seeing previously:

 .─────────.     ┌─────────┐
(  fb75aff  )◀───│changelog│
 `─────────'     └─────────┘
     │
     │   .─────────.    ┌────┐   ┌────┐
     │  (  bece83c  )◀──│main│◀──│HEAD│
     │   `─────────'    └────┘   └────┘
     │        │
     │        ▼
     │   .─────────.
     │  (  3bbdb69  )
     │   `─────────'
     │        │
     │┌───────┘
     ▼▼
 .─────────.
(  c8c57bf  )
 `─────────'

Compare this to what git log --graph --all is showing you. Can you identify the arrows pointing to each commit’s parent? Can you see where the branches and the HEAD are pointing to?

You can ask git to display this graph in an extremely compact form:

$ git log --format=oneline --graph --all
* fb75affebd2d44fd7100bfe932a06004623825cf (changelog) Added changelog to README.txt
| * bece83c61b9d0bd58f5d7d90cb9ca10af8771802 (HEAD -> main) Added an empty file
| * 3bbdb697b2fe653de8f73c9071bcacb3aaa3b79b Added second line to README.txt
|/
* c8c57bfe49e8d5d1918571f4cbdc708cfd82501e Initial commit, added README.txt

Even though there is no arrow shown between bece83c and 3bbdb69, you can still see that 3bbdb69 is the parent of bece83c.

We will talk more about git log in coming articles. But for now, try this exercise:

Remove the --all option from the command. What happens?
Switch back to the changelog branch, and then display the graph with and without the --all option.
Switch to a commit in a detached HEAD state, and inspect the graph again.
What happens if you don’t include the --graph option?

Common Lisp development with Vim

gajon@gajon.org (Jorge Gajon) — Tue, 07 Dec 2010 00:00:00 +0000

You have used Vim for so long, and you have it so ingrained within you, that when somebody suggests you use other editor you instinctively reply “over my dead body!”. I know how you feel.

And then one day you start programming in Common Lisp; everybody says that you must use Emacs with SLIME; and after not so long you actually start considering switching to it!

The horrors!

I believe that for some long-time Vim users, switching to Emacs is not an easy option. Personally I’m afraid of immersing into Emacs, spending countless hours learning its ways and idiosyncrasies, without gaining any significant advantage over Vim. Those countless hours could be better spend immersing yourself into learning Common Lisp (or something else).

In this article I’ll show you how I’m using Vim to develop applications with Common Lisp.

There are a few plug-ins for Vim to do Common Lisp development, two of them are Limp and slimv. Of those two I liked Limp better, although it does some things in ways that I don’t agree with, therefore I modified it a little to meet my needs. Keep in mind that those are my personal preferences, and I don’t claim them to be the correct way of doing things.

But before continuing, I’d like to express my gratitude and admiration to Limp’s author, Mikael Jansson, for all the hard work put into it.

Now I’m going to show you the modifications I did to Limp, but first you need to download and install it; it shouldn’t be too difficult, follow the documentation.

1. Disable highlighting.

Limp does automatic highlighting of contents inside any pair or parenthesis, but I think there’s two problems with it. First, I find it distracting, I don’t like the constant changing of colors all over the place while moving the cursor through the code. Second, it slows down the rendering of the text significantly, and the whole thing feels unresponsive.

I don’t see the benefit of this highlighting. I think that keeping a sane indentation of your code is enough to see the nested structures in it. I also think that if you don’t keep a logical indentation of the code at all times, you are doing something wrong.

We will also see how to make Vim indent Lisp code correctly further below.

Inside the main Limp folder there is a vim subfolder; open the file limp.vim and at the end you can see several runtime commands; comment out the one that loads the highlight.vim file.

"runtime ftplugin/lisp/limp/highlight.vim

Then open the file mode.vim and around line number 58 you can see the call to initialize the highlighting mode; you need to comment it out too.

"call LimpHighlight_start()

And the call to stop the mode, located around line number 68, comment it out too (if you are using an SVN version of the code, it may already be commented).

"call LimpHighlight_stop()

2. Disable Autoclosing of parenthesis.

Another feature of Limp is an automatic insertion of closing parenthesis when you type the opening parenthesis. There’s a big problem with this feature in that it is buggy and leaves visual debris on the screen when undoing operations (at least with gvim).

Even if it were not buggy, I think that far from helping it will only distract you unnecessarily. You will have a huge number of closing parenthesis after your cursor while in the middle of trying to figure out the shape of your code. And you still need to type the closing parenthesis on your keyboard to continue writing the next form.

I believe that you only need the highlighting of matching pairs, automatic indentation, and a shortcut to automatically close any remaining open parenthesis once you are satisfied with your block of code

We’ll see how to add such a shortcut further below.

If you want something to manage parenthesis that is similar to Emacs’ ParEdit you could probably find a plugin that does that (I didn’t look up).

Like in the previous step, comment the instruction to load this feature inside the file limp.vim:

"runtime ftplugin/lisp/limp/autoclose.vim

And inside mode.vim comment the function call to initialize the mode:

"call AutoClose_start()

And of course the call to stop the mode needs to be commented out too:

"call AutoClose_stop()

3. Restore your own color scheme.

I don’t like the colors that Limp sets up, and I don’t see why it should change them in the first place. If you want to continue using your own colorscheme, open the file mode.vim and around line number 30 comment these lines:

"set t_Co=256
"if !exists("g:colors_name")
    "colorscheme desert256
"endif

You can see that there’s a bunch of highlight group definitions there. These are definitions for when using Vim in a terminal, and since I mostly only use gvim I didn’t have to do anything with them. But if you do use Vim inside a terminal you may want to adjust these colors to match your colorscheme, or simply comment them out and use the defaults.

4. Better options

I didn’t like the set of options that Limp sets, especially the old Vi Lisp indentation mode. I also dislike folding so I disabled that too. My current set of options look like this (file mode.vim around line 82):

syntax on
setlocal nocompatible nocursorline
setlocal lisp syntax=lisp
setlocal ls=2 bs=2 et sw=2 sts=2 ts=8 tw=80
setlocal statusline=%<%f\ \(%{LimpBridge_connection_status()}\)\ %h%m%r%=%-14.(%l,%c%V%)\ %P
"setlocal iskeyword=&,*,+,45,/,48-57,:,<,=,>,@,A-Z,a-z,_
"setlocal cpoptions=-mp
"setlocal foldmethod=marker foldmarker=(,) foldminlines=1
setlocal foldcolumn=0

set lispwords+=defgeneric,block,catch,with-gensyms

"-----------
"Taken from the bundled lisp.vim file in VIM
"(/usr/share/vim/vim72/ftplugin/lisp.vim)
setl comments=:;
setl define=^\\s*(def\\k*
setl formatoptions-=t
setl iskeyword+=+,-,\*,/,%,<,=,>,:,$,?,!,@-@,94
setl comments^=:;;;,:;;,sr:#\|,mb:\|,ex:\|#
setl formatoptions+=croql
"-----------

" This allows gf and :find to work. Fix path to your needs
setlocal suffixesadd=.lisp,.cl path+=/home/gajon/Lisp/\*\*

Notice I set tw=80 (you may want to disable it); modified the statusline to be less verbose; disabled folding; disabled the cpoptions line because I want Vim’s default; copied the options that come bundled with Vim (they are a lot better); added some symbols to lispwords; and added a missing dot to suffixesadd (cl extension was missing a dot).

It is common to find files edited by Emacs (and other editors) that uses a mixture of spaces and tabs to indent code, and usually they use tabs for 8 spaces and hard spaces for the rest of the indenting if it is not a multiple of 8. That’s the reason why I set ts=8, so that you can see those files with appropriate indentation.

5. Disabling the transposing of sexps.

Limp binds they keys { and } to functions that transpose the current sexp with the previous and next sexp. But they don’t work reliably and I think they are unnecessary when you can just as easily use dab and p at the proper place. Besides, the default Vim {} bindings are quite useful to jump to other top-level forms.

In file keys.vim comment these lines:

"nmap <buffer> {                    <Plug>SexpMoveBack
"nmap <buffer> }                    <Plug>SexpMoveForward

6. Bug when connecting to a running REPL

There’s a bug that prevents Limp from reconnecting to an already running REPL. In file bridge.vim inside the vim subfolder, around line number 13:

let cmd = s:Limp_location . "/bin/lisp.sh ".core_opt." -s ".styfile." -b ".name

A space was missing between ".core_opt." and -s.

If you obtained an SVN version of Limp, this have been fixed already.

7. Let’s add some improvements, a better REPL

The preceding changes were adjustments that I believe work best for me. You may want to follow them or do other modifications, it’s basically all a personal preference. In the rest of this article I’ll show you a few small additions that I made to Limp.

When you press <F12> Limp will launch an xterm and start SBCL inside it; this is done from the lisp.sh file in the bin folder.

Let’s make the SBCL REPL able to complete symbols from the thesaurus that Limp generated when you installed it (see Limp’s install docs). This is done by telling rlwrap where to look for a list of words to consider for auto completion.

Around line 149 in file lisp.sh we can see the line that defines the parameters to rlwrap.

[[ `which rlwrap` ]] && RLWRAP="rlwrap -b $BREAK_CHARS"

We can modify that line like this:

[[ `which rlwrap` ]] && RLWRAP="rlwrap -b $BREAK_CHARS -f $LIMPDIR/vim/thesaurus"

Now when Limp starts an SBCL REPL, you can start typing a symbol and hit <Tab> to complete it; for example if you start typing get-u and press <Tab>, rlwrap will complete it to get-universal-time.

But you can add more options to rlwrap, for example I have this:

[[ `which rlwrap` ]] && RLWRAP="rlwrap -pgreen -r -s 2000 -m -i -c -b $BREAK_CHARS -f $LIMPDIR/vim/thesaurus"

The -pgreen option makes rlwrap colorize the prompt to green (see man rlwrap for available colors); the -s 2000 option is to make rlwrap remember 2000 lines of input; the -m option enables multiline editing (see below); the -i turns case sensitivity off, and the -c option enables you to complete filenames.

The -r option makes rlwrap remember all words seen in the input and output streams; this means that, in addition to those symbols found in the thesaurus, it will also be able to complete symbols you typed in the REPL.

You can edit the input by calling an external editor defined by the environment variable $RLWRAP_EDITOR, and that editor can of course be Vim; this allows us to do multi-line editing.

Around line 154 in the same file, we can see the actual invocation to SBCL:

$RLWRAP $SBCL --noinform $core

We can modify that line to introduce the $RLWRAP_EDITOR environment variable:

RLWRAP_EDITOR="vim -c \"set ft=lisp\" +%L" $RLWRAP $SBCL --noinform $core

To invoke the editor you can hit CTRL-^, which the man page of rlwrap says is the default binding but that didn’t work for me and I had to explicitly add it to my ~/.inputrc file.

My ~/.inputrc (see man rlwrap and man readline):

$include /etc/inputrc
set editing-mode vi
tab: complete
set completion-ignore-case on
set blink-matching-paren on
"\C-^":rlwrap_call_editor

$if sbcl
    set comment-begin ;
$endif

I would also suggest that you load the sb-aclrepl module, which will give you a better REPL prompt and an inspector. To enable it edit your ~/.sbclrc file and add this form:

(require :sb-aclrepl)

You can find more info here http://www.sbcl.org/manual/#sb_002daclrepl. After you have loaded it you can type :help at the prompt to see a list of available commands. You an also skim the following web page to understand some of the commands simulated by sb-aclrepl http://www.franz.com/support/documentation/6.2/doc/top-level.htm

8. Additional mappings

Now let’s add a few useful mappings, you shouldn’t have any problem figuring out how to use these mappings:

In the file bridge.vim add the following lines after line number 265:

nnoremap <silent> <buffer> <Plug>EvalUndefine   :call LimpBridge_send_to_lisp("(fmakunbound '".expand("<cword>").")")<CR>
nnoremap <silent> <buffer> <Plug>EvalAddWord    :let &lispwords.=',' . expand("<cword>")<cr>

nnoremap <silent> <buffer> <Plug>DebugTrace         :call LimpBridge_send_to_lisp("(trace ".expand("<cword>").")")<CR>
nnoremap <silent> <buffer> <Plug>DebugUnTrace       :call LimpBridge_send_to_lisp("(untrace ".expand("<cword>").")")<CR>
nnoremap <silent> <buffer> <Plug>DebugInspectObject :call LimpBridge_inspect_expression()<CR>
nnoremap <silent> <buffer> <Plug>DebugInspectLast   :call LimpBridge_send_to_lisp("(inspect *)")<CR>
nnoremap <silent> <buffer> <Plug>DebugDisassemble   :call LimpBridge_send_to_lisp("(disassemble #'".expand("<cword>").")")<CR>
nnoremap <silent> <buffer> <Plug>DebugMacroExpand   :call LimpBridge_macroexpand_current_form( "macroexpand" )<CR>
nnoremap <silent> <buffer> <Plug>DebugMacroExpand1  :call LimpBridge_macroexpand_current_form( "macroexpand-1" )<CR>

nnoremap <silent> <buffer> <Plug>ProfileSet      :call LimpBridge_send_to_lisp("(sb-profile:profile ".expand("<cword>").")")<CR>
nnoremap <silent> <buffer> <Plug>ProfileUnSet    :call LimpBridge_send_to_lisp("(sb-profile:unprofile ".expand("<cword>").")")<CR>
nnoremap <silent> <buffer> <Plug>ProfileShow     :call LimpBridge_send_to_lisp("(sb-profile:profile)")<CR>
nnoremap <silent> <buffer> <Plug>ProfileUnSetAll :call LimpBridge_send_to_lisp("(sb-profile:unprofile)")<CR>
nnoremap <silent> <buffer> <Plug>ProfileReport   :call LimpBridge_send_to_lisp("(sb-profile:report)")<CR>
nnoremap <silent> <buffer> <Plug>ProfileReset    :call LimpBridge_send_to_lisp("(sb-profile:reset)")<CR>

And at the end of the file we add these two functions:

function! LimpBridge_inspect_expression()
  let whatwhat = input("Inspect: ")
  call LimpBridge_send_to_lisp( "(inspect " . whatwhat . ")" )
endfun

function! LimpBridge_macroexpand_current_form(command)
  " save position
  let pos = LimpBridge_get_pos()

  " find & yank current s-exp
  normal! [(
  let sexp = LimpBridge_yank( "%" )
  call LimpBridge_send_to_lisp( "(" . a:command . " '" . sexp . ")" )
  call LimpBridge_goto_pos( pos )
endfunction

As you can see we added a bunch of internal mappings which we will use to send forms to the REPL, by using the function LimpBridge_send_to_lisp(). We also added two new functions, LimpBridge_inspect_expression() which will prompt you for a form and inspect its result in the REPL. The other function LimpBridge_macroexpand_current_form() will call macroexpand or macroexpand-1 on the form under the cursor.

To actually use these new mappings we have to map them to some keys by adding the following lines inside the file keys.vim:

nmap <buffer> <LocalLeader>eu      <Plug>EvalUndefine
nmap <buffer> <LocalLeader>ea      <Plug>EvalAddWord

nmap <buffer> <LocalLeader>dt      <Plug>DebugTrace
nmap <buffer> <LocalLeader>du      <Plug>DebugUnTrace
nmap <buffer> <LocalLeader>di      <Plug>DebugInspectObject
nmap <buffer> <LocalLeader>dI      <Plug>DebugInspectLast
nmap <buffer> <LocalLeader>dd      <Plug>DebugDisassemble
nmap <buffer> <LocalLeader>ma      <Plug>DebugMacroExpand
nmap <buffer> <LocalLeader>m1      <Plug>DebugMacroExpand1

nmap <buffer> <LocalLeader>pr      <Plug>ProfileSet
nmap <buffer> <LocalLeader>pu      <Plug>ProfileUnSet
nmap <buffer> <LocalLeader>pp      <Plug>ProfileShow
nmap <buffer> <LocalLeader>pa      <Plug>ProfileUnSetAll
nmap <buffer> <LocalLeader>ps      <Plug>ProfileReport
nmap <buffer> <LocalLeader>p-      <Plug>ProfileReset

Now, assuming that your “leader” key is the inverted slash you can type \eu when the cursor is over a symbol to undefine it, by calling fmakunbound on it. And it’s similar with the rest of the mappings we just defined; you can consult the SBCL manual to learn how to use the tracing functionally as well as the profiler.

One mapping that is not connected to REPL functionality is EvalAddWord mapped to the keys \ea. I use this to add symbols to Vim’s lispwords option. When you write your own macros, unless you add its name to lispwords, Vim will indent all forms passed to it as if they were arguments to a regular function.

As an example, suppose you wrote a macro:

(defmacro my-macro (foo &body body)
  ... body of the macro ...)

Vim will indent code that calls this macro like this:

(my-macro hello
          (+ 10 20))

But after adding the macro name to lispwords by positioning the cursor over my-macro and hitting \ea, Vim will indent the code like this (hit \ft to indent the top form):

(my-macro hello
  (+ 10 20))

9. Closing open parenthesis

I told you at the beginning of this article that having a shortcut to auto-close open unbalanced parenthesis was desirable. Let’s add such shortcut. I extracted the following from the slimv plugin, written by Tamas Kovacs.

Let’s add a new mapping inside the file sexp.vim, around line number 53:

nnoremap <silent> <buffer> <Plug>SexpCloseParenthesis  :call SlimvCloseForm()<CR>

and in the same file, at the very bottom, we’ll add these two functions:

"-------------------------------------------------------------------
" Close open parenthesis
" Taken from the Slimv plugin by Tamas Kovacs. Released in the
" public domain by the original author.
"-------------------------------------------------------------------

" Count the opening and closing parens or brackets to determine if they match
function! s:GetParenCount( lines )
    let paren = 0
    let inside_string = 0
    let i = 0
    while i < len( a:lines )
        let inside_comment = 0
        let j = 0
        while j < len( a:lines[i] )
            if inside_string
                " We are inside a string, skip parens, wait for closing '"'
                if a:lines[i][j] == '"'
                    let inside_string = 0
                endif
            elseif inside_comment
                " We are inside a comment, skip parens, wait for end of line
            else
                " We are outside of strings and comments, now we shall count parens
                if a:lines[i][j] == '"'
                    let inside_string = 1
                endif
                if a:lines[i][j] == ';'
                    let inside_comment = 1
                endif
                if a:lines[i][j] == '(' || a:lines[i][j] == '['
                    let paren = paren + 1
                endif
                if a:lines[i][j] == ')' || a:lines[i][j] == ']'
                    let paren = paren - 1
                    if paren < 0
                        " Oops, too many closing parens in the middle
                        return paren
                    endif
                endif
            endif
            let j = j + 1
        endwhile
        let i = i + 1
    endwhile
    return paren
endfunction

" Close current top level form by adding the missing parens
function! SlimvCloseForm()
    let l2 = line( '.' )
    normal 99[(
    let l1 = line( '.' )
    let form = []
    let l = l1
    while l <= l2
        call add( form, getline( l ) )
        let l = l + 1
    endwhile
    let paren = s:GetParenCount( form )
    if paren > 0
        " Add missing parens
        let lastline = getline( l2 )
        while paren > 0
            let lastline = lastline . ')'
            let paren = paren - 1
        endwhile
        call setline( l2, lastline )
    endif
    normal %
endfunction

Now we only need to add a key mapping inside keys.vim right after where we added the previous mappings

nmap <buffer> <LocalLeader>cp      <Plug>SexpCloseParenthesis
imap <buffer> <C-X>0               <C-O><LocalLeader>cp

Now you can type \cp to close open parenthesis, but notice that I also added an imap to be able to type CTRL-X 0 while in insert mode as well; that’s what I usually use.

10. That’s all

I hope this helps you make a better use of Limp. When I started learning Common Lisp I didn’t want to be distracted by trying to simultaneously learn Emacs and SLIME. I used Vim and Limp exclusively during my first project with Common Lisp, and I didn’t have any major problems.

For my second project - and already feeling comfortable with Common Lisp - I decided to use only Emacs and SLIME. And I have to say that yes, SLIME is a superior development environment for Common Lisp. It is not indispensable, you can develop software without it just fine, but it is indeed a useful tool.

However, the editing model of Emacs is too limiting, and I think that if I want to continue using it with SLIME, I’ll have to explore options like viper or vimpulse; it’s just too painful not to have the Vi editing model.

Addendum: My window environment

I use a window manager called Ion, which is designed to be used with the keyboard instead of the mouse. Another nice feature of Ion is that the windows never overlap (you could make them overlap though) and they are positioned in tiles that you define.

This means that I can, for example, position my code in the left half of my screen and a REPL on the right half, without any effort. I don’t have to struggle with resizing and moving windows so that I can see them both at the same time, as would be the case with a traditional window manager.

Here’s a screenshot of how my screen looks:

~
~
:wq

Common Lisp presentation at the HackerRoom MX

gajon@gajon.org (Jorge Gajon) — Wed, 10 Nov 2010 00:00:00 +0000

The HackerRoom MX is a co-working space for hackers in Mexico City. I gave a Common Lisp talk this last September to a few of the inhabitants there.

I’m putting the slides at http://gajon.org/otherstuff/lisp-slides/

If you are creating a similar presentation feel free to take whatever you want from my slides. I hereby authorize You to do whatever the hell you want with the content referred to above. Except that I took some parts from other places (always showing a link to the original source). If you use something from those parts you should include the link to the original sources as well.

Enjoy.

Review of "Eric Sink on the Business of Software"

gajon@gajon.org (Jorge Gajon) — Fri, 09 Apr 2010 00:00:00 +0000

Eric Sink on the Business of Software
Eric Sink
Apress (March 20, 2006)

This is a book targeted to people (more specifically, programmers) who are thinking about starting a software business, or are working in a startup or small software business.

The author is the founder of SourceGear, an Illinois based company which develops and sells version control tools for developers, and through his own experiences he shows us the lessons he has learned; many times by making a lot of mistakes.

The book is divided in four parts; (1) Entrepreneurship, (2) People, (3) Marketing, and (4) Sales. In each part the author answers the most common questions most developers have when thinking about becoming an entrepreneur.

He also tries to demystify many topics that are nebulous and obscure to most programmers, especially the ones that relate to marketing, sales and finances. He shows us that these topics are not as complex as they may seem and he even provides us with algorithms that we can use to better understand them.

The nice thing about this book is that the author, not being a “business” person, but a programmer at heart, writes from that perspective, making it easy to relate to him, and see that he had the same questions, self-doubts and fears that most programmers have.

The content of this book is taken from blog articles the author has published on his own website, with some comments added at the beginning of each one. This means that (1) you could read everything on his web site (though I prefer reading from a book), and (2) there is some repetition of ideas that were said in a previous article.

Highly recommended reading if you have dreams of starting your own software business someday.

Review of "ANSI Common Lisp"

gajon@gajon.org (Jorge Gajon) — Wed, 10 Mar 2010 00:00:00 +0000

ANSI Common Lisp
Paul Graham
Prentice Hall (November 12, 1995)

This is a really good book for newcomers to Common Lisp, however take note that this is not a book for beginning programmers. This book will not guide you on the details of how to get, install and launch a Common Lisp environment. If you don’t feel able to do that on your own then this book might not be for you.

The pace of the book is fast, and the information is presented in a very compressed form, meaning that the author does not ramble unnecessarily on any topic. The concepts are introduced one after another, many times building new concepts on top of the ones that have been introduced before; in this sense this book is meant to be read in order.

To give you an idea of the compactness of the book, the 17 chapters of the book (not counting the appendixes) add up to only 285 pages.

There are exercises at the end of each chapter, with varying degrees of difficulty. Personally, I would’ve liked it better if there were more exercises that required the student to construct more levels of abstraction to find a solution.

The topics of CLOS and Macros receive a fair treatment on this book, but not an exhaustive one. For further study into these two topics I would suggest the books “Object-Oriented Programming in Common Lisp: A Programmer’s Guide to CLOS” by Sonya E. Keene, and “On LISP: Advanced Techniques for Common LISP” also by Paul Graham.

Although the latter book is out of print and very hard to find a copy of, a PDF version is offered by the author at his website http://www.paulgraham.com/onlisp.html

One thing I like about this book is its thorough treatment of cons objects and how they are used in Common Lisp, along with discussions on destructive and non-destructive (side-effect free) functions, and the issues of sharing structure.

Other highlights are, a simple implementation of a Ray-tracer; an implementation of a custom Object-Oriented language within Common Lisp; an implementation of a program that makes inferences from a set of rules (very similar to Prolog); and utilities to generate HTML.

Finally, it’s got a reference of all the language at the end of the book, which makes it very convenient to take the book with you, away from the computer, and try understanding the topics and do the exercises using only pen and paper. In fact, I recommend you do that, and use the computer only to verify that the solutions you found are correct.

It took me 48 hours over the space of 39 days (averaging nearly 1:15 hrs a day) to read this book and do the exercises.

Conclusion: Highly recommended as an introductory book in Common Lisp.

Trees as Linked Lists in Common Lisp

gajon@gajon.org (Jorge Gajon) — Tue, 23 Feb 2010 00:00:00 +0000

If you are starting to learn Common Lisp, and have already read the chapter about “Cons” from an introductory book (also called cons cells), then you could try the following exercise. How to represent a tree using nothing but conses; and abstract that implementation with functions to create and traverse the tree.

PLEASE NOTE, that if you need to represent trees in a production program you shouldn’t use lists as described here unless you have a good reason. This is only an exercise in understanding how cons cells work.

1. A Tree

Suppose we would like to represent the following tree in memory, using only lists (which are created by chaining conses). You should already know how a cons is created and what parts constitute one. If not, go back to an introductory book on Common Lisp.

This tree can be represented as a set of linked lists, like in the following diagram:

The natural way to represent the previous tree as a Common Lisp list is like the following:

(1 (2 6 7 8) 3 (4 (9 12)) (5 10 11))

The first element of a list is the root of that tree (or a node) and the rest of the elements are the subtrees (more nodes). For example, the subtree (2 6 7 8) is a tree with “2” at its root and the elements “6 7 8” as its children.

Exercise

How would you draw a diagram of the cons cells of the previous list? Compare it to the figure below.

The cons cells that build up the list above can be drawn like this:

2. Another representation of the Tree

The list representing the tree shown above is intuitive and easy to understand, however there’s a slight inconvenience. A subtree can be represented as a cons that directly contains the data in it’s car position, like in the case of “3”, or as a cons that refers to another cons that starts a proper list, like in the case of (2 6 7 8).

There’s no consistency, when you need to access the data contained in a node, you must first check if the car position of the cons refers to a list, in which case you have to access that list to get the data.

Also, in the example list above, we are storing numbers in the nodes of the tree. What if you wanted to reference any kind of data from a node?, including a list?

Exercise

How would you represent a single node of a tree, so that the methods to access the data, the children, and the next sibling are always the same?, how could the node reference any kind of data, including other lists?

Compare it to the representation given below.

There could be more than one way to represent a node, but in the rest of this small article we are going to represent them like in the figure above.

A tree node consists of two conses, the cdr of the first cons refers to the siblings of the node, while the cdr of the second cons refers to the subtrees (children) of the node. The car of the first cons refers to the second cons, while the car of the second cons refers to the data stored in this node, which can be any kind of object.

Using this node representation, the example tree given in Figure 1 would be displayed by Common Lisp as the following list. Can you see why?

((1 (2 (6) (7) (8)) (3) (4 (9 (12))) (5 (10) (11))))

Exercise

How would you draw a diagram of the cons cells of the previous list? Compare it to the figure below.

3. The API

These are the functions that we are going to define for creating and traversing the nodes of a tree.

(defun make-tree (data)
  "Creates a new node that contains 'data' as its data."
  ...)

(defun add-child (tree child)
  "Takes two nodes created with 'make-tree' and adds the
  second node as a child of the first. Returns the first node,
  which will be modified."
  ...)

(defun first-child (tree)
  "Returns a reference to the first child of the node passed in,
  or nil if this node does not have children."
  ...)

(defun next-sibling (tree)
  "Returns a reference to the next sibling of the node passed in,
  or nil if this node does not have any siblings."
  ...)

(defun data (tree)
  "Returns the information contained in this node."
  ...)

Notice that the function add-child will modify the node passed as the first argument, it is a destructive function, it has side effects.

If you haven’t read about destructive or side-effecting functions, then you should go back and continue reading your book on Common Lisp.

Exercise

Given the structure of a node as shown in Figure 4 above, come up with the code for the functions outlined above.

Compare your code to the solutions given below.

Test your code with the following usage example:

(let ((one (make-tree 1)))
  (add-child one (make-tree 2))
  (add-child one (make-tree 4))
  (add-child one (make-tree 5))
  (let ((two (first-child one)))
    (add-child two (make-tree 6))
    (add-child two (make-tree 7))
    (let ((four (next-sibling two)))
      (add-child four (make-tree 9))
      (add-child (first-child four) (make-tree 12))))
  one)

=> ((1 (2 (6) (7)) (4 (9 (12))) (5)))

Code for the functions outlined above

I’m going to skip showing a code for the function add-child for now, we are going to discuss it in the next section.

The first function make-tree just needs to create the two conses as depicted in Figure 4, and place the data in the correct place.

(defun make-tree (data)
  "Creates a new node that contains 'data' as its data."
  (cons (cons data nil) nil))

Similarly, for the other functions, we just need to follow the correct cells as suggested in Figure 4.

(defun first-child (tree)
  "Returns a reference to the first child of the node passed in,
  or nil if this node does not have children."
  (cdr (car tree)))

(defun next-sibling (tree)
  "Returns a reference to the next sibling of the node passed in,
  or nil if this node does not have any siblings."
  (cdr tree))

(defun data (tree)
  "Returns the information contained in this node."
  (car (car tree)))

Easy, is it not?. Were you checking for null trees? That is, where you doing something like this?

(defun first-child (tree)
  (if (null tree)
    nil
    (cdr (car tree))))

(defun first-child (tree)
  (if (null (cdr (car tree)))
    nil
    (cdr (car tree))))

Well, don’t, that’s Java/C++ thinking. Notice that (car nil) evaluates to nil, and similarly, (cdr nil) evaluates to nil too.

However, trying to pass an atom or other object that is not a cons to either car or cdr is an error. For example (first-child 3) would cause an error.

You could try to guard against that redefining first-child like this:

(defun first-child (tree)
  (when (listp tree)
    (cdr (car tree))))

However, I wouldn’t do that, because I think that signalling an error on a function call like (first-child 3) is the correct thing to do. The user of the function is the one that is committing an error, not the implementor of the function.

4. First approach to add-child

There are a few ways you could have implemented the function add-child, let’s consider an example that at first sight might look fine.

(defun add-child (tree child)
  (setf (car tree) (append (car tree) child))
  tree)

Remember that the cdr of the second cons (car tree) refers to the children of the node, so that when we do (append (car tree) child) we are appending child to the end of the list of conses that start with the one in (car tree). Since append does not modify any of its arguments and instead returns a fresh “consed” list (a new copy), we have to capture it again with (setf (car tree) ...).

By the way, the setf line can be replaced with this line,

(rplaca tree (append (car tree) child))

rplaca stands for “replace car”, and it replaces the car of the first argument with whatever the second argument evaluates to.

Now, let’s try building the whole tree depicted in Figure 1; the following code can accomplish that.

(let ((one (make-tree 1)))
  (add-child one (make-tree 2))
  (add-child one (make-tree 3))
  (add-child one (make-tree 4))
  (add-child one (make-tree 5))
  (let ((two (first-child one)))
    (add-child two (make-tree 6))
    (add-child two (make-tree 7))
    (add-child two (make-tree 8)))
  (let ((four (next-sibling (next-sibling (first-child one)))))
    (add-child four (add-child (make-tree 9) (make-tree 12)))
    (let ((five (next-sibling four)))
      (add-child five (make-tree 10))
      (add-child five (make-tree 11))))
  one)

=> ((1 (2 (6) (7) (8)) (3) (4 (9 (12))) (5 (10) (11))))

By visually inspecting the resulting list you can see that the tree is being built correctly. But you might wonder why juggle with first-child and next-sibling with nested let bindings, why not create the nodes that we need to refer more than once in the first let binding.

(let ((one    (make-tree 1))
      (two    (make-tree 2))
      (four   (make-tree 4))
      (five   (make-tree 5)))
  (add-child one two)
  (add-child one (make-tree 3))
  (add-child one four)
  (add-child one five)
  (add-child two (make-tree 6))
  (add-child two (make-tree 7))
  (add-child two (make-tree 8))
  (add-child four (add-child (make-tree 9) (make-tree 12)))
  (add-child five (make-tree 10))
  (add-child five (make-tree 11))
  one)

=> ((1 (2) (3) (4) (5 (10) (11))))

What??, the children of nodes “2” and “4” were lost, but somehow the children of node “5” were added correctly. There’s no difference in the way we create nodes 2, 4 and 5, and there’s no difference in how we add children to them either. What’s going on?

Exercise

This is where you will really test your understanding of how conses work and how references to objects in Common Lisp work. Try to figure out why the function add-child as used above is failing to correctly build the tree that we want.

Drawing the cons cells after each operation will be helpful.

You will also need to be sure to understand how append works.

5. Understanding what is going wrong

To understand what is going wrong, let’s see what happens after let creates the bindings, but before any form inside the let body is evaluated.

We have four variables bound to four tree nodes. Then after evaluating the form (add-child one two) we have the following objects.

The function append does not modify any of its arguments, and therefore to be able to build a new list it has to create new conses copying the top list structure of all the lists it has been passed in as arguments, except for the very last list, which will be just pointed to by the second to last list.

After (append (car tree) child) has created the new list, we point to it by doing (setf (car tree) ...), thereby losing the reference to the original cons that was there (shown in grey in the above figure).

Please Note, the figure above may create a confusion. I’ve been drawing the numbers inside the cons for simplicity, but this is not very accurate. The two components of a cons (the car and the cdr) can refer to any type of object, and these objects may live in other parts of the memory.

Sometimes an implementation may store integers and other data types directly inside the cons, but you shouldn’t care or depend on this.

Functions like append only copy the “list structure”, it will never copy the data that a cons refers to. (But make sure to understand the difference between append and copy-tree, refer to your Common Lisp book or the HyperSpec.)

A more accurate depiction of the cons cells and the objects they refer to would look like this.

However, I’ll continue drawing the numbers inside the cons for simplicity.

OK, so moving on, after the form (add-child one (make-tree 3)) has been evaluated, we have the following situation.

append copied the top list structure of the first list it had as an argument, this new list points to the last list, the node “3”. “Top list structure” means that append will only copy the conses that it reaches by following the cdrs of each cons; it will never go down the car of any cons (that’s what copy-tree would do).

The conses of the new list are shown in blue, while the original cons that was holding the value “1” and pointing to TWO is shown in grey. Since nothing is pointing to this grey cons, it may be garbage collected.

Now let’s see what happens after evaluating (add-child one four).

Again, we see the new copies of conses marked in blue, while the original conses are marked in grey. Since those grey conses can’t be reached anymore, they may eventually be garbage collected.

And, after evaluating (add-child one five) we have the following.

Again, the blue conses are the copies made by append and the grey conses here were the blue conses of Figure 10. Can you see why there’s only three grey conses (which can be GC) and four new blue conses?

So far everything seems to be working ok. But now comes the interesting part.

Exercise

From the state of cons cells depicted in Figure 11 above, can you see what will happen after the form (add-child two (make-tree 6)) is evaluated?, can you see where lies the problem?. Drawing the conses could be helpful.

Compare it to the figure below.

Let’s recall the definition of add-child:

(defun add-child (tree child)
  (setf (car tree) (append (car tree) child))
  tree)

We can see that we are calling append on (car tree) and child, which correspond to (car TWO) and the value of (make-tree 6). Therefore it’s easy to see that append is creating a copy of the cons in (car TWO), returning a new list. Finally, setf sets the (car TWO) to point to this new list, completely separating TWO from the original tree starting with ONE.

Now we have two disjoint trees, one starting at the node pointed to by ONE and the other starting at the node pointed to by TWO. If we keep adding children to TWO, they will not be reachable from ONE. Likewise if we add children to the node FOUR.

Can you see why there’s no problem in adding children to the node FIVE?

6. A correct version of add-child

The problem with the definition of add-child above is that while it is defined to modify the argument tree, it’s actually only modifying part of it, and making new copies of other parts of tree (its children).

In other words, we should consider that the argument tree refers not only to a single node, but to a whole “tree” with subtrees. Of course the fact that we named the argument tree is to remind us of that.

We have to be always careful when designing our functions, and be clear if they will modify their arguments and how; or otherwise specify that our function will be completely side-effect free.

When we say that the function add-child will modify the first argument to add the second argument as a child of it, then it must do that, but do it to the whole conceptual tree.

The other alternative is to build a side-effect free add-child, which will create a new copy of the whole tree with the child appended, and leave the responsibility of capturing that new tree and updating any and all references to it or parts of it, to the user of our function.

So how do we correct add-child to do the correct thing?. We know that we can’t use append because it creates new copies of the children that tree had. Knowing that, the solution is easy, we just need to use the destructive version of append, which is called nconc.

nconc won’t create any copy of anything, and instead will modify it’s arguments so that they are linked in a single list, much like what append does.

(defun add-child (tree child)
  "Takes two nodes created with 'make-tree' and adds the
  second node as a child of the first. Returns the first node,
  which will be modified."
  (nconc (car tree) child)
  tree)

nconc will modify the cdr of the second cons of tree to add the child if it had not previous children; otherwise it will modify the last node in the list of children of tree, and link the child to it.

Can you see why it’s just (nconc (car tree) child) and there’s no need to use setf?

But be very careful, nconc is the only destructive function for which you don’t need to capture it’s result with setf. When you are using other destructive functions like nreverse you must always capture the result with setf.

7. Traversing the tree

Finally, it’s difficult to see the returned list and interpret it. Let’s try to create a function to traverse the tree as the last exercise.

(let ((one    (make-tree 1))
      (two    (make-tree 2))
      (four   (make-tree 4))
      (five   (make-tree 5)))
  (add-child one two)
  (add-child one (make-tree 3))
  (add-child one four)
  (add-child one five)
  (add-child two (make-tree 6))
  (add-child two (make-tree 7))
  (add-child two (make-tree 8))
  (add-child four (add-child (make-tree 9) (make-tree 12)))
  (add-child five (make-tree 10))
  (add-child five (make-tree 11))

  ;; Print the contents of the tree,
  (traverse one)
  ;; and return it.
  one)

If we input the above into the REPL, we should get the following output.

Data: 1  Children: (2 3 4 5)
   Data: 2  Children: (6 7 8)
      Data: 6
      Data: 7
      Data: 8
   Data: 3
   Data: 4  Children: (9)
      Data: 9  Children: (12)
         Data: 12
   Data: 5  Children: (10 11)
      Data: 10
      Data: 11
((1 (2 (6) (7) (8)) (3) (4 (9 (12))) (5 (10) (11))))

Notice that the list ((1 (2 ...) is the value of one returned by the let form, while the information before it is being printed by (traverse one).

Exercise

Write the function traverse so that it generates the output shown above. Compare it to the solution given below.

Tip: you can use the "~v@T" directive to the format function to move the cursor forward. For example:

(format t "~v@TFoo" 2)
=>  Foo

(format t "~v@TBar" 4)
=>    Bar

(format t "~v@TFoo" 6)
=>      Foo

Let’s build the solution in steps. First we should print the node data:

(format t "~&~v@TData: ~A" padding (data tree))

The parameter padding will tell format to indent the line a certain number of columns. We’ll see how to calculate that later. The "~&" directive tells format to print a newline only at the beginning of a line.

We also need to print the children of the node, on the same line, but only if there are children to print.

(format t "~&~v@TData: ~A" padding (data tree))
(when (first-child tree)
  (format t "  Children: ~A" (first-child tree)))

But wait, that’s not correct, it would print the whole tree of the children of node, like this:

Data: 1  Children: ((2 (6) (7) (8)) (3) (4 (9 (12))) (5 (10) (11)))

We only want the data in the nodes of the direct children of the node, as a list. We can use the maplist function for that. maplist applies a function to successive sublists of a list.

(format t "~&~v@TData: ~A" padding (data tree))
(when (first-child tree)
  (format t "  Children: ~A"
          (maplist #'(lambda (x) (data x))
                   (first-child tree))))

Now let’s wrap this in a function.

(defun traverse (tree &optional (padding 0))
  (format t "~&~v@TData: ~A" padding (data tree))
  (when (first-child tree)
    (format t "  Children: ~A"
            (maplist #'(lambda (x) (data x))
                     (first-child tree)))))

OK, now that we have printed the data of the node and a list of the children, we have to repeat this process for each of the children of this node, and they should be printed indented below the current node.

Since we already have the code to print the node and a list of its children, we will reuse this function and do a recursive call. One important thing to keep in mind when writing a recursive function is write a test for ending the recursion. In this case the recursion must end when the parameter tree is nil.

(defun traverse (tree &optional (padding 0))
  (when tree
    (format t "~&~v@TData: ~A" padding (data tree))
    (when (first-child tree)
      (format t "  Children: ~A"
              (maplist #'(lambda (x) (data x))
                       (first-child tree))))
    (traverse (first-child tree) (+ padding 3))))

We can see above how the padding is being incremented, in this case by 3 columns. Since the parameter padding is optional, in the first invocation (traverse one) the padding will be zero, and then it will be incremented by 3 on each recursive invocation.

But wait, the above code will only print the leftmost nodes in the tree, we still need to print the siblings of the nodes. That is very easy.

(defun traverse (tree &optional (padding 0))
  (when tree
    (format t "~&~v@TData: ~A" padding (data tree))
    (when (first-child tree)
      (format t "  Children: ~A"
              (maplist #'(lambda (x) (data x))
                       (first-child tree))))
    (traverse (first-child tree) (+ padding 3))
    (traverse (next-sibling tree) padding)))

Since the siblings should be printed at the same level as the current node, the padding is left unmodified.

:EOF

And that’s it. I hope this helped you get a better grasp of how conses work.

Other uses for OpenSSH

gajon@gajon.org (Jorge Gajon) — Sun, 31 May 2009 00:00:00 +0000

Many people use OpenSSH to connect to a remote machine, but many don’t know that you can actually do other interesting things with it, other than just opening a remote login shell. In this article I’ll show you a few of them.

We will see how to:

Copy files from one machine to another securely.
Forward ports from your machine through the remote machine, creating a secure tunnel for your unsecured applications.
Increase the security of your OpenSSH communications.

Make sure the OpenSSH daemon is running.

OpenSSH is the most widely used implementation of SSH, and you’ll probably find it already installed in most UNIX like systems.

To be able to connect to the remote machine you’ll need to have the sshd daemon running in it. If this remote machine is actually physically remote then it must have it already running, otherwise how could you do anything at all?.

But if your remote machine is sitting right next you, or you are running it as a virtual machine and you are experimenting with OpenSSH, then you can find out if the daemon is running by executing the following command:

$ ps -ef | grep sshd

You should see a line similar to this (the numbers will vary):

root      2859     1  0 13:29 ?        00:00:00 /usr/sbin/sshd

If it is not already running consult your distribution documentation to find out how to to start it. On an Slackware system you can do that by executing the following command as root:

# /etc/rc.d/rc.sshd start

To have the sshd daemon run every time your Slackware system boots up, you’ll have to chmod the startup script so that it has execute permission.

# chmod 755 /etc/rc.d/rc.sshd

On a Debian system you can start the daemon by executing the following command:

# /etc/init.d/ssh start

To find out how to make it start every time your Debian system boots up, read the update-rc.d man page to figure out how.

$ man update-rc.d

1. Copy files from one machine to another securely.

1.1. Using `scp` for simple transfers.

You can easily copy files from one machine to another through an encrypted connection with OpenSSH, and this will prevent anyone else from spying what you are sending over. The basic form is:

mymachine:~$ scp somefile.tar gajon@remote.com:

That will copy the file somefile.tar to the remote machine remote.com at the home folder of the user gajon (I’ll be using my own user name in all the examples here, but of course you should use your own instead).

It is important to put a colon (:) after the remote machine name, otherwise scp will just rename your local file as gajon@remote.com in the same local folder in your machine.

It’s also possible to specify the path and final name that you want the file to have in the remote server. For example, this will copy the file to the folder backups that is located in the user’s home.

mymachine:~$ scp somefile.tar gajon@remote.com:backups/

This will copy the file to the same folder as the previous example, but will save the file with a different name.

mymachine:~$ scp somefile.tar gajon@remote.com:backups/back02.tar

You can also copy files from the remote machine to your local machine, you just have to reverse the order of the arguments. For example, this will copy the file myapp.log that is located in the /var/log folder of the remote machine to your local machine, but with the name remote.log instead.

mymachine:~$ scp gajon@remote.com:/var/log/myapp.log remote.log

If you wanted to preserve the original name you could have used the following command:

mymachine:~$ scp gajon@remote.com:/var/log/myapp.log .

Notice that the remote path after the colon starts with a slash, that indicates an absolute path. If you don’t start the path with a slash then it’s a relative path, relative to the home of the user in the remote machine.

1.2 Using `sftp` for secure FTP transfers.

Many people know how to use FTP, but some may not realize that everything they transfer during an FTP session, including their user names and passwords, are sent unencrypted or protected in any form. This opens up the possibility that someone with ill intentions might capture that information.

sftp solves that problem and it is used the same way as ftp, so that the user doesn’t need to learn anything new, and most graphical FTP clients can also use sftp – for example, FileZilla.

In this example we get the file back02.tar from the remote machine and save it in our local computer.

mymachine:~$ sftp gajon@remote.com
Connecting to remote.com...
gajon@remote.com's password:
sftp> cd backups
sftp> get back02.tar
sftp> bye

When you log on to a remote machine with the sftp client the prompt will change to sftp>. The commands are almost the same as with a traditional ftp, you can enter ? to see a list of available commands.

2. Forwarding ports.

2.1. From your local machine to a remote machine.

Port forwarding is useful for creating a secure data tunnel for an application that doesn’t support encryption or other form of secure communication out of the box. This is also useful to go through a firewall and skip any of its restrictions.

As an example, lets suppose you want to access a POP3 server. You can redirect one of your local ports, lets say 9999, to the port 110 (POP3) of the remote machine.

mymachine:~$ ssh -L 9999:localhost:110 gajon@remote.com

Next you need to configure your email client to use port 9999. OpenSSH will take all traffic on port 9999 in your local machine and send it encrypted to the remote machine, where the data will continue through port 110 to its original destination.

Notice that the remote machine will send the data to the final destination using port 110, AND this data will NOT be encrypted anymore from that point on. You must be aware of that, if your remote machine and the destination machine (POP3 server) are the same then it’s OK, otherwise there’s still a risk.

2.2. From the remote machine to your local machine.

It’s also possible to do a forwarding in the other direction, from the remote machine to your local machine. An example of when this could be useful is when you have a machine in your office that sits behind a firewall that blocks all inbound connections, but permits all outbound connections, and you want to be able to connect to your office machine from your home and open an ssh session.

From the office machine you can establish the tunnel to the home machine:

office:~$ ssh -R 2222:localhost:22 gajon@home

This tunnel will listen on port 2222 of the home machine and will encrypt data and send it to port 22 of the machine at the office. Once you are at home you can connect to the office machine from home with OpenSSH, using this tunnel that you established from the office machine.

home:~$ ssh -p 2222 gajon@office

Notice that we are indicating to ssh that we want to connect with the port 2222 (-p option), this is because that’s the port that your local machine is listening to.

2.3. Dynamic port forwarding.

Dynamic port forwarding lets you forward any port on your local machine through a secure tunnel to a remote machine, by creating a SOCKS server.

Let’s say that you have a restrictive firewall at the office that blocks web pages that have not been authorized. Imagine you need to find a solution to a programming problem you are having but can’t do that because the firewall is blocking all websites that contain anything similar to a forum, and we all know that forums are where we always find solutions to a technical, undocumented problem.

Well, that has happened to me, and the solution is a SOCKS server that can proxy our data through a remote machine; and have it encrypted as an added bonus!.

You can establish a dynamic port forwarding tunnel as:

mymachine:~$ ssh -D 12345 gajon@remote.com

With that command, you establish a connection to the remote machine, but OpenSSH will listen to any SOCKS communications on your local machine at port 12345.

To be able to use this tunnel, your application that needs to connect must know how to use the SOCKS protocol (either SOCKS4 or SOCKS5). On Firefox it’s easy to do that, we go to Preferences and in the Advanced group select the Network tab, click on the Settings button and select Manual proxy settings and in the SOCKS Server field put localhost, and port 12345.

And that’s it, when Firefox needs to load a website, it will communicate with the SOCKS server in your local machine with port 12345; OpenSSH will encrypt your data and send it through a secure channel to the remote machine, where the data will continue its way to its destination and original port.

But there are many applications that don’t implement a SOCKS protocol, one example is Opera. There are tools that can wrap a non SOCKS application and transparently send all its connections through a SOCKS server. One such application that I have used is tsocks.

3. Increase the security of your OpenSSH communications.

There are several things you can do to increase the security of your OpenSSH communications, and it’s a good idea to do them.

3.1. Disable root logins.

The most useful and easiest way to increase the security of your OpenSSH server is to disable root logins. To do this you must edit the sshd_config file in your server.

remote:~# vim /etc/ssh/sshd_config

Search for the PermitRootLogin attribute and change it to no. Then restart your OpenSSH server. On Slackware you use the following line:

remote:~# /etc/rc.d/rc.ssh restart

On a Debian system you use the following line:

remote:~# /etc/init.d/sshd restart

3.2. Change the port where OpenSSH listens.

There are many people with bad intentions that use automated tools that try to breach OpenSSH servers. If you have a server on the Internet, you may have noticed in your logs that there are lots and lots of attempts to login to your server using many different user names. What they are trying to do is to get to one server by applying a so called dictionary attack, that is, attempting a huge combination of common user names and passwords, with the hope that one of those combinations will actually succeed.

A very easy way to stop that is to simply change the port of your OpenSSH daemon. If the attacker cannot find a server at the default port 22 he will probably just move to the next server that he can find. These kind of attackers don’t bother checking every port of the machine, that would take too much time, instead what they do is try all the ip addresses they can.

To change the port number of your server, edit the sshd_config file again and change the Port attribute. For example:

Port 30234

Restart the OpenSSH daemon, and now to connect you have to specify the port to use. You can use the -p switch:

mymachine:~$ ssh -p 30234 gajon@remote.com

But having to type the -p switch and the port is tedious, so instead of that you can specify the port in a config file that your ssh client can use. This config file is usually found in the $HOME/.ssh/ folder.

Open up the $HOME/.ssh/config file in your editor and put these lines there:

Host remote.com
  Port 30234

Make sure the Port line below the Host is indented.

This way you don’t have to specify the port on the command line.

mymachine:~$ ssh gajon@remote.com

You could also add the user name in the config file:

Host remote.com
  User gajon
  Port 30234

And then you can just type:

mymachine:~$ ssh remote.com

3.3. Specify which groups can connect.

You can tell OpenSSH that only those users belonging to a certain group can log in. For example, lets create a group sshers, and add your user to that group, you do that with root. In this example I’ll be adding the user gajon to the group.

We have to do this on the remote server, and the first thing to do is create the group:

remote:~# groupadd sshers

Then check what groups the user is already a member of:

remote:~# groups gajon
gajon : users wheel audio video cdrom games

Then make it a member of the sshers group:

remote:~# usermod -G wheel,audio,video,cdrom,games,sshers gajon
remote:~# groups gajon
gajon : users wheel audio video cdrom games sshers

Notice that the first group listed with groups gajon is users, that’s the primary group of the user, the rest are the secondary groups. You modify the list of secondary groups with usermod -G, separating each group name with a coma. See man usermod for more details.

OK, now that you have your new group and your user is part of that group, let’s configure the OpenSSH daemon so that only members of that group can connect at the remote server. Edit the sshd_config file in the server:

remote:~# vim /etc/ssh/sshd_config

And add the AllowGroups property to to file:

# Only members of group sshers will be able to connect
AllowGroups sshers

And that’s it, any user that is not part of the sshers group will not be able to log in.

3.4. Use Public-Key Authentication.

It is not a good idea to type the password of your remote user while connecting to the remote machine. The problem is that if someones guesses your password, or looks over your shoulder while you are typing it, or somehow it is compromised, that person that has your password can easily connect to the remote machine from anywhere.

To avoid this danger, you could instead authenticate by setting up a so called “Public-Key Cryptography” authentication.

In Public-Key Cryptography, each party has a pair of keys, one key is known as the “private key” while the other is known as the “public key”. The idea is that when you want to communicate with another party you use their public key and only that who has the corresponding private key can decrypt the message.

This topic of Public-Key Cryptography is quite interesting, if you want to read more you can read the Wikipedia page about it.

The OpenSSH daemon can be configured to authenticate users with a Public-Key Cryptography method. The advantage of this scheme is that you will have your pair of keys, one public and one private, and if someone managed to obtain your user password it won’t matter, because no one will be able to log in to the remote system without the proper private key needed to establish the authentication.

Also, for an attacker to be able to log in into your system, not only he’ll need to get your private key, but also the password you use to decrypt your private key, which is only inside your head.

Let’s create your key pair in your local machine:

mymachine:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/gajon/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/gajon/.ssh/id_rsa.
Your public key has been saved in /home/gajon/.ssh/id_rsa.pub.
The key fingerprint is:
fd:6b:f8:82:85:40:e2:8b:1b:31:47:c6:94:25:47:87 gajon@mymachine

Two keys have been generated, one private (id_rsa), and the other public (id_rsa.pub). The password that you entered is used to encrypt your private key.

Notice that the files were placed in your $HOME/.ssh/ folder in your computer. Make sure that’s the case, you don’t want them scattered on your file system.

Next you have to copy your public key to the $HOME/.ssh/authorized_keys file in the remote server. The authorized_keys file can contain the information of several public keys, so to avoid overwriting an existing authorized_keys in your remote server, you are going to append the contents of your public key to the authorized_keys file. And if that file did not exist, it will be created anyway.

mymachine:~$ cat .ssh/id_rsa.pub | ssh gajon@remote.com "cat >> .ssh/authorized_keys"

Now to connect to the remote machine using Public-Key Cryptography authentication, you need to use the -i switch and specify the private key to use:

mymachine:~$ ssh -i ~/.ssh/id_rsa gajon@remote.com
Enter passphrase for key '/home/gajon/.ssh/id_rsa':
remote.com:~$

Notice that your ssh client is asking you for your private key password, which is NOT the same password for your system user, but the password you entered when you created this pair of keys.

After you have seen that public key authentication works, it is a good idea to disable simple password logins on your remote machine, so that the only way of connecting to it is with the use of Public-Key Cryptography. To do that you’ll need to set the PasswordAuthentication property with the value of no, in the sshd_config file.

remote.com:~# vim /etc/ssh/sshd_config

Disable tunnelled clear text passwords:

PasswordAuthentication no

Restart your sshd daemon, and now you will only be able to log in to the remote server if the user in the remote server has the public key info in the .ssh/authorized_keys file, AND you have the corresponding private key in your own machine, and of course you know the password to that private key.

BE VERY CAREFUL, after you make the above change and restart the daemon, do not log out of your session just yet, open up another terminal and try to make a second connection to your remote machine. If you can’t, try to figure it out in the other session that you still have open. I’m telling you this because if you disable password authentication, and the public key authentication fails, you could easily lock yourself out of the remote machine, which would be really bad.

Now, it is annoying having to specify the private key to use with the -i switch every time you want to connect to the remote machine. So instead of that, you can configure what private key to use in the $HOME/.ssh/config file in your local machine.

Retaking the example we saw in section 3.2, you add the IdentityFile setting which specifies a private key to use when connecting to the host.

Host remote.com
  User gajon
  IdentityFile ~/.ssh/id_rsa
  Port 30234

This way it’s not necessary to specify the private key on the command line, nor the user or the port (if it were different from the default 22):

mymachine:~$ ssh remote.com
Enter passphrase for key '/home/gajon/.ssh/id_rsa':
remote.com:~$

A final note; you can name your keys any way you want. If you have more than one remote machine you should give more meaningful names to your key pairs, so that you can more easily manage them. For example, if we had three servers, “breakpoint”, “lucretia” and “she-wolf”, we could create a $HOME/.ssh/config file with the following entries:

Host breakpoint
  HostName 11.22.33.44
  User gajon
  IdentityFile ~/.ssh/id_rsa_breakpoint
  Port 30234

Host lucretia
  HostName lucretia.example.com
  User johnny
  IdentityFile ~/.ssh/id_rsa_lucretia

Host she-wolf.remote.com
  User admin
  IdentityFile ~/.ssh/id_rsa_shewolf

Notice that I added another setting, HostName, which specifies the actual host to connect to. It could be an IP address or a fully qualified name of the server. That way the Host setting is actually an alias that you can freely define. With that example we can easily connect to the “breakpoint” server, with an IP address of 11.22.33.44.

mymachine:~$ ssh breakpoint

And that’s all for now.

I hope these tips are useful to you, but remember that we have just seen a few uses of OpenSSH, and that there are a lot more. For example, in a next article I’ll write how to use Public-Key authentication to easily issue commands to remote servers and automate tasks.

Going to PyCon 2008

gajon@gajon.org (Jorge Gajon) — Mon, 10 Mar 2008 00:00:00 +0000

There’s only 4 days left before I fly to Chicago to attend the PyCon 2008 conference, the biggest event for the Python community. This will be the second time that I go to PyCon, last year the conference was held in Addison Texas and I really had a great time, I met cool people, visited a few restaurants and learned interesting things about Python.

Here are some photos that I took last year, and you can see more on my flickr set:

I’m sure this year’s PyCon will be great!

Update: PyCon 08 is over, here are some of the photos that I took.

It’s nice to meet intelligent people and be surrounded by all this enthusiasm around this cool language and all the projects that are being built with it.

Installing a Realtek RTL8185 wireless card

gajon@gajon.org (Jorge Gajon) — Mon, 25 Feb 2008 00:00:00 +0000

In this post I explain how I made my wireless card with a Realtek RTL8185 chipset work in Slackware GNU/Linux.

I recently moved to a new apartment and the Internet modem had to be placed in the bedroom, separate from the living room where I have my workstation, so I had to choose between buying a wireless card for my pc or drilling a hole through the wall to slip in an ethernet cable.

Since I don’t have a drill, and my wife wouldn’t have liked a cable going through the middle of the makeup mirror, I got an Encore Electronics Wireless-G PCI Adapter (ENLWI-G2). Actually this is just a card with a Realtek RTL8185 chipset on it, and you can get the Linux drivers from the Realtek website.

Building the modules and testing

I have to warn you that I had my PC crash a few times while experimenting with the wireless settings, so be patient and be prepared to hit the power button.

Download and extract the appropriate *.tar.gz file from the Realtek website, you will find instructions on how to build and load the modules in the readme file. Basically you have to:

Run the file makedrv to build the modules from the source code.
Run the file wlan0up to load the modules into the running kernel.

This is where I encountered the first problem, for some reason the last line in the wlan0up script (ifconfig wlan0 up) made my computer crash. After rebooting I commented out that line and tried again, no problem was found and I saw that the wlan0 interface was already up anyway (run ifconfig to see the interfaces that are up).

These modules are not going to be loaded automatically at boot up yet, we’ll see how to do that later.
The third step after the wireless interface is up is to configure your wireless link settings and getting an IP address, see the readme file to get the details.

In my case I only needed these two commands to set up the wireless link to the router.

iwconfig wlan0 essid "Megadeth"
iwconfig wlan0 key abcd123456

My wireless LAN name is “Megadeth” and it uses WEP encryption in Open security mode. Yes, I know that WEP is practically useless for security, but I have other devices that only talk WEP.

After setting the wireless link I had to run this command to get an ip address from the router:

dhcpcd -t 20 wlan0

I did NOT run the wlan0dhcp file that was provided in the tarball as it appears to be RedHat specific, or something like that, I really don’t know, but I know for sure that it wouldn’t work in Slackware.

Making the modules load at boot up

The next step was to figure out how to have the modules loaded up when the computer starts.

One option is to leave the compiled modules where they are (or move it wherever you want) and load them up by putting these instructions at the end of the /etc/rc.d/rc.modules-XYZ script where XYZ is the version of the kernel that the modules were built for:

/sbin/insmod /root/rtl8185/ieee80211/ieee80211_crypt-rtl.ko
/sbin/insmod /root/rtl8185/ieee80211/ieee80211_crypt_wep-rtl.ko
/sbin/insmod /root/rtl8185/ieee80211/ieee80211_crypt_tkip-rtl.ko
/sbin/insmod /root/rtl8185/ieee80211/ieee80211_crypt_ccmp-rtl.ko
/sbin/insmod /root/rtl8185/ieee80211/ieee80211-rtl.ko
/sbin/insmod /root/rtl8185/rtl8185/r8180.ko

The /root/rtl8185/ folder is where I unpacked and built the modules.

The second option, which is the one I prefer, is to move the modules to the appropriate kernel directories so that you don’t clutter your root folder.

But before that you should know that the ieee80211 modules that are built from this package are intended as a replacement for the ieee80211 stack that comes with the kernel. This means that you have to move out the original ieee80211 stack to avoid having them loaded into the kernel. This might not be strictly necessary since the modules have different names, but I wasn’t using the original stack and I didn’t see any need for it so I moved them out just as a precaution, as having both stacks loaded would have caused a conflict.

# Back up the original stack.
cd /lib/modules/2.6.21.5-smp/kernel/net
mv ieee80211 /root/ieee80211_original_stack

# Move the new modules there.
# we are at /lib/modules/2.6.21.5-smp/kernel/net
mkdir ieee80211
cp /root/rtl8185/ieee80211/*.ko ieee80211/

# Finally move the r8180.ko module too.
cd /lib/modules/2.6.21.5-smp/kernel/drivers/net/wireless
cp /root/rtl8185/rtl8185/r8180.ko .

# Update module dependencies.
depmod -ae

The last command, depmod -ae, updates a file /lib/modules/2.6.21.5-smp/modules.dep which indicates the dependencies for each module. If you inspect that file you’ll see that the r8180 module depends on the ieee80211_rtl and ieee80211_crypt_rtl modules; indeed, if you reboot and type lsmod you’ll see that these modules are loaded.

r8180
ieee80211_rtl
ieee80211_crypt_rtl

However, there are three additional modules that need to be loaded too:

ieee80211_crypt_wep-rtl
ieee80211_crypt_tkip-rtl
ieee80211_crypt_ccmp-rtl

Without these modules you won’t be able to set encryption keys for your wireless link. These are not being loaded because no module depends on them. To change that we have to edit the modules.dep file and edit the line that specifies the dependencies of the r8180 module, the format of a modules.dep entry is:

/path/to/module_a.ko: /path/to/module2.ko /path/to/module1.ko

Which means that module_a.ko depends on module1.ko and module2.ko. The modules are loaded from right to left, which means that module1 is loaded first, followed by module2 and finally module_a last.

So then, open up the modules.dep file and search for the line where the dependencies of the r8180 are defined and change it so that all the necessary modules are loaded.

Here I break the line into multiple lines so that you can read it easily, but make sure it is only one whole line, so please pay attention. Also, the order is important, so pay double attention.

I repeat, this must be one single line in your modules.dep file:

/lib/modules/2.6.21.5-smp/kernel/drivers/net/wireless/r8180.ko:
/lib/modules/2.6.21.5-smp/kernel/net/ieee80211/ieee80211-rtl.ko
/lib/modules/2.6.21.5-smp/kernel/net/ieee80211/ieee80211_crypt_ccmp-rtl.ko
/lib/modules/2.6.21.5-smp/kernel/net/ieee80211/ieee80211_crypt_tkip-rtl.ko
/lib/modules/2.6.21.5-smp/kernel/net/ieee80211/ieee80211_crypt_wep-rtl.ko
/lib/modules/2.6.21.5-smp/kernel/net/ieee80211/ieee80211_crypt-rtl.ko

After you reboot you’ll see that all the modules have been loaded up.

Setting the wireless link and getting an IP at boot up

The final step is to set your wireless essid and key and get an ip from the router when your computer boots up.

Since I run Slackware, the place to define ip settings is the file /etc/rc.d/rc.inet1.conf - the relevant lines in my settings file are:

...
IFNAME[4]="wlan0"
USE_DHCP[4]="yes"
DHCP_TIMEOUT[4]=30
WLAN_ESSID[4]=Megadeth
WLAN_KEY[4]="abcd123456 open"
...

There are a few things to notice here.

I added the variable DHCP_TIMEOUT, this is because the /etc/rc.d/rc.inet1 script executes an ifconfig wlan0 up if that variable is not defined (look around line 117), and I don’t know why, but for some reason sometimes that makes my computer freeze.

This is strange because the rc.wireless script also executes this command but I have not found any problem with it, it seems that the card doesn’t like to be “upped” so many times.

For the WLAN_ESSID variable you’ll type your wireless LAN name without quotes.

You’ll notice that the WLAN_KEY contains the word open, this is because if you don’t specify either open or restricted the /etc/rc.d/rc.wireless script will set the key to restricted mode by default. In my case I wanted it to be in open security mode.

One last thing that is important, the /etc/rc.d/rc.wireless script sets a nick option on the wireless settings, but this driver does not support that option and it will throw an error. Open that file and comment out that line, look around line 184, i.e.:

if [ ! -n "$NICKNAME" ] ; then
    NICKNAME=`/bin/hostname`
fi
if [ -n "$ESSID" -o -n "$MODE" ] ; then
    echo "$0:  $IWCOMMAND nick $NICKNAME" | $LOGGER
    # $IWCOMMAND nick $NICKNAME <-- this is not supported
fi

And that’s it, try running /etc/rc.d/rc.inet1 stop and then /etc/rc.d/rc.inet1 start.

You’ll notice that this line is printed on the screen:

./rc.wireless:  wlan0 information: 'Any ESSID...'

If it bothers you, you can edit the /etc/rc.d/rc.wireless.conf file and comment out the lines from line number 38 to 41 and you won’t see that message anymore.

Or, you can change the INFO variable to something else. I’m more nostalgic and so I changed it to a quote from an old song I like:

## --------- START SECTION TO REMOVE -----------
## Pick up any Access Point, should work on most 802.11 cards
*)
    INFO="The warheads will all rust in peace"
    ESSID="Megadeth"
    ;;
## ---------- END SECTION TO REMOVE ------------

It’s not necessary to put the ESSID here as it is overridden by what you set on the rc.inet1.conf file.

If things get really nasty….

If you screwed it with the wireless settings and you can’t get your pc to boot up normally because the driver crashes it, you can boot up in single user mode (also called emergency mode). When you boot up your computer and get the LILO prompt, select your Linux installation and type the word single after the label.

In single user mode no networking configurations will be set up at all so that you can edit your settings and try again.

Enjoy…

Never walk alone...

git tips and tricks - Part 1: the fundamentals

What is a git repository?

Branches are pointers to commits

Going back to our main branch

Display the repository graph with git log

Common Lisp development with Vim

1. Disable highlighting.

2. Disable Autoclosing of parenthesis.

3. Restore your own color scheme.

4. Better options

5. Disabling the transposing of sexps.

6. Bug when connecting to a running REPL

7. Let’s add some improvements, a better REPL

8. Additional mappings

9. Closing open parenthesis

10. That’s all

Addendum: My window environment

Common Lisp presentation at the HackerRoom MX

Review of "Eric Sink on the Business of Software"

Review of "ANSI Common Lisp"

Trees as Linked Lists in Common Lisp

1. A Tree

Exercise

2. Another representation of the Tree

Exercise

Exercise

3. The API

Exercise

Code for the functions outlined above

4. First approach to add-child

Exercise

5. Understanding what is going wrong

Exercise

6. A correct version of add-child

7. Traversing the tree

Exercise

:EOF

Other uses for OpenSSH

Make sure the OpenSSH daemon is running.

1. Copy files from one machine to another securely.

1.1. Using scp for simple transfers.

1.2 Using sftp for secure FTP transfers.

2. Forwarding ports.

2.1. From your local machine to a remote machine.

2.2. From the remote machine to your local machine.

2.3. Dynamic port forwarding.

3. Increase the security of your OpenSSH communications.

3.1. Disable root logins.

3.2. Change the port where OpenSSH listens.

3.3. Specify which groups can connect.

3.4. Use Public-Key Authentication.

And that’s all for now.

Going to PyCon 2008

Installing a Realtek RTL8185 wireless card

Building the modules and testing

Making the modules load at boot up

Setting the wireless link and getting an IP at boot up

If things get really nasty….

Display the repository graph with `git log`

1.1. Using `scp` for simple transfers.

1.2 Using `sftp` for secure FTP transfers.