Inserting a new commit tree into Git history

Git does an excellent job of making it easy to moved change sets around in the history via git rebase. Most of the time this is exactly what you need to do when altering history.

However, sometimes you want to insert a complete tree at a particular point in the history. You already have exactly that tree somewhere else in a different commit. Since Git actually works in terms of entire trees underneath (compressing them using deltas for storage only) this isn't particularly hard to do.

I needed to do this with a branch we used for storing upstream releases that we receive as tarballs. The history looks like this:

* b9a5c3889cb Import from linux-4.1-1.12.tar.bz2
* 0e9df32da3f Import from linux-4.1-1.9.tar.bz2
* 714ffa13302 Import from linux-4.1-1.6.tar.bz2
* 5d33c6e6665 Import from linux-4.1-1.3.tar.bz2
* 0434412e517 Import from linux-4.1-1.2.tar.bz2

Now suppose that before we've merged b9a5c3889cb we receive a linux-4.1.7.tar.bz2 tarball and want to insert it into the history. So, let's check out 714ffa13302 and import the tarball as usual.

git checkout 714ffa13302
rm -rf *
tar xjf /tmp/linux-4.1-1.7.tar.bz2
git add -A .
git commit -m "Import from linux-4.1.7.tar.bz2"

So, now we have:

* 31124b678e5 Import from linux-4.1-1.7.tar.bz2
* 714ffa13302 Import from linux-4.1-1.6.tar.bz2
* 5d33c6e6665 Import from linux-4.1-1.3.tar.bz2
* 0434412e517 Import from linux-4.1-1.2.tar.bz2

If we try and rebase b9a5c3889cb on top of 31124b678e5 then we'll just get a huge amount of conflicts as Git tries to apply the differences rather than the tree.

Instead we need to tell Git to construct a new commit with the correct parent:

new_parent=31124b678e5
transplanted_commits="0e9df32da3f..b9a5c3889cb"

git rev-list --reverse ${transplanted_commits} | while read commit; do
    echo "Dealing with ${commit}"
    new_parent=$(git cat-file commit ${commit} |
    awk '/^$/ {emit=1} { if (emit) print }' |
    git commit-tree -p ${new_parent} ${commit}^{tree} -F -)
    echo "new_parent ${new_parent}"
done

git commit-tree prints the new commit hash on stdout, so after running this you can just read out the last new_parent and run something like:

git checkout -b fixed-branch <last-new_parent>

and the history on fixed-branch will have been rewritten to contain the extra new commit.

The awk magic is required to remove the header information from the commit message before reusing it for the replacement commit.

Of course, better shell scripting could turn this into a more generic tool, but this was good enough for me.

I found the git commit-tree trick in the answers to this StackOverflow question.

i3 with Gnome underpinnings on Debian 8.0 (Jessie)

I've been running i3 as my window manager on Debian 7.0 (Wheezy) for a while. I meant to get round to writing a blog posting about how I set it up to work somewhat sanely with Gnome, GDM and gnome-session but never got round to it. The upgrade to Jessie brought some new problems as Gnome rolls more core functionality into gnome-shell. So, here's what I've done to create a Gnome+i3 session in GDM.

Create /usr/share/xsessions/i3-gnome.desktop containing:

[Desktop Entry]
Encoding=UTF-8
Name=i3 Gnome
Comment=improved dynamic tiling window manager with Gnome
TryExec=/usr/local/bin/gnome-session-i3
Exec=gnome-session-i3
Type=Application

Create /usr/local/bin/gnome-session-i3 containing:

#!/bin/sh
exec gnome-session --session=i3-gnome --debug "$@"

and ensure it is executable with:

chmod +x /usr/local/bin/gnome-session-i3

Create /usr/share/gnome-session/sessions/i3-gnome.session containing:

[GNOME Session]
Name=i3 Gnome session
RequiredComponents=gnome-settings-daemon;gnome-screensaver;nm-applet;i3-gnome

Create /usr/share/applications/i3-gnome.desktop containing:

[Desktop Entry]
Type=Application
Encoding=UTF-8
Name=i3-gnome
Comment=improved dynamic tiling window manager with Gnome underpinnings
Exec=i3-gnome
NoDisplay=true
X-GNOME-WMName=i3 Gnome
X-GNOME-Autostart-Phase=WindowManager
X-GNOME-Provides=windowmanager
StartupWMClass=i3

Create /usr/local/bin/i3-gnome containing:

#!/bin/sh
if [ -n "$DESKTOP_AUTOSTART_ID" ]; then
   dbus-send --session --print-reply --dest=org.gnome.SessionManager "/org/gnome/SessionManager" org.gnome.SessionManager.RegisterClient "string:i3-gnome" "string:$DESKTOP_AUTOSTART_ID"
fi
exec i3

and ensure it is executable with:

chmod +x /usr/local/bin/i3-gnome

That should give you enough to have an "i3 Gnome" entry in your GDM menu. But, if you launch it you'll find there are a few problems that need solving: To make the mouse cursor appear (credit):

dconf write /org/gnome/settings-daemon/plugins/cursor/active false

In order to log out you need to tell the session manager to finish the session. I do that with a script that provides the behaviour that gnome-session-quit used to in earlier Gnome versions:

#!/bin/sh

# gnome-session-quit talks DBus to gnome-session which now tries to
# talk to gnome-shell in order to bring up a dialog box to confirm
# whether we should quit. Since we're not running gnome-shell this
# doesn't work. So, let's bring up our own box and then tell
# gnome-session to quit without prompting.

case "$1" in
    --logout)
 text="Do you wish to log out?"
 ;;
    --power-off)
 text="Do you wish to power off?"
 ;;
    --reboot)
 text="Do you wish to reboot?"
 ;;
    *)
 echo "Unknown option: $1" 1>&2
 exit 1
esac

if zenity --question --text="${text}"; then
    exec gnome-session-quit --no-prompt "$@"
fi

and then bind it in ~/.i3/config:

bindsym $mod+Shift+E exec /home/mac/bin/scripts/gnome-session-quit-ask --logout

In order to lock the session using gnome-screensaver you can add a binding like this to your .i3/config:

bindsym Ctrl+Mod1+L exec --no-startup-id dbus-send --print-reply --session --dest=org.gnome.ScreenSaver /org/gnome/ScreenSaver org.gnome.ScreenSaver.Lock

Fast user switching works with this but when switching back to i3 you'll unfortunately need to enter your password twice. I use udisks-glue to automatically mount storage devices.

How to stop Windows+P reconfiguring your monitor with Gnome3

I've recently switched to using i3-wm as my Window manager under Gnome3 on Debian 7.0 (Wheezy.) I'm using the Windows key (Mod4) for most of my key bindings and it's taking a bit of getting used to. In particular I have trouble when I've just typed something involving the Windows key and I now need to press the Alt key. This means I keep accidentally pressing Windows+P which causes all my displays to change configuration. Fixing this situation is time consuming since I need to move my containers back onto the correct displays.

So, I went looking for a configuration setting with:

gsettings list-recursive|grep -i '<super>p'

Which revealed that I could easily disable this behaviour with:

gsettings set org.gnome.settings-daemon.plugins.media-keys video-out ""

So now when I press Windows+P all I get is a far less serious "p" character. This might be problematic if I had a laptop that generated the Windows+P key combination when pressing the monitor select function key but I don't so that isn't a problem for me.

mp3chop 2.0 released with AAC support

I added support for ADTS AAC to mp3chop over a year ago. I wanted to support M4A files too but since I didn't manage to get round to it I thought I'd better release what I had. You can download it from the mp3chop website or look at the Git repository.

ssh-and-tmux: part three

On many of the hosts I connect to using my ssh-and-tmux script I want to be able to use git to connect to repositories via ssh. Initially this is as simple as ensuring that ssh-agent forwarding is enabled by passing -A to ssh. The problem comes when I disconnect and reconnect from somewhere else. This kills the ssh session and the socket used for the agent is removed and SSH_AUTH_SOCK ends up pointing to a file that doesn't exist.

The solution is to use a fixed name for the authentication socket and symlink that name to the real one as it changes.

#!/bin/sh
set -e

if [ -z "$SSH_AUTH_SOCK" ]; then
    echo No ssh agent found. Starting one.
    eval `ssh-agent`
fi

if [ -n "$SSH_AUTH_SOCK" ]; then
    export SSH_AUTH_SOCK_OLD=$SSH_AUTH_SOCK
    export SSH_AUTH_SOCK=$HOME/.ssh/.tmux-agent
    ln -sf "$SSH_AUTH_SOCK_OLD" $SSH_AUTH_SOCK
else
    echo Failed to start ssh agent

    # Set the socket anyway in case someone reattaches later with a
    # valid agent.
    export SSH_AUTH_SOCK=$HOME/.ssh/.tmux-agent
fi

# Use a specific name for the tmux/screen session so
# that we can tell that it's one that uses this scheme.
# We have to detach other clients with tmux so that we
# don't end up with a stale agent in them when we
# return to them later.
if which tmux >/dev/null 2>/dev/null; then
    tmux -2 -L ssh-agent "$@" attach -d || tmux -2 -L ssh-agent "$@"
else
    exec screen -S ssh-agent "$@" -x -RR
fi

Now the original script just needs to run this script on the remote host rather than executing tmux directly.

I've been using this final solution for quite a while now and it works really well.

ssh-and-tmux: part two

Many of the hosts I wish to use my ssh-and-tmux script to connect to are behind firewalls so I can't connect to them directly. I find that it is more reliable to just ssh through a gateway host rather than using any VPN that might be available. This can easily be bolted on to the ssh-and-tmux script provided a good guess can be made as to which network you're actually on:

#!/bin/sh
if [ -n "$TMUX" ]; then
    echo Already in tmux
    exit 1
fi

if [ -n "$STY" ]; then
    echo Already in screen
    exit 1
fi

while true; do
    # First work out where we are based on our IP address.
    # We do this every time round the loop in case we've 
    # moved network since last time.
    addrs=`ip --family inet --oneline addr`

    work_via=gateway.example.com
    home_via=gateway.randombitsofuselessinformation.blogspot.com

    case "$addrs" in
 *192.168.1.*)
     # We're on the home network
     home_via=
     ;;
 *172.16.*)
     # We're on the work network
     work_via=
     ;;
    esac

    extra=
    via=
    case "$1" in
 work-host1)
     via=$work_via
     # Add a port forward for this host too
     extra=-L8080:work-host1:8080
     dest=work-host1
     ;;
 home-host1)
     via=$home_via
     dest=home-host1
     ;;
 home-host2)
     via=$home_via
     dest=home-host2
     ;;
 *.*)
     # Hosts with dots are assumed to be on the
            # Internet at large
     via=
     dest="$1"
     ;;
 *)
     # All other hosts are assumed to be on the work
            # network
     via=$work_via
     dest="$1"
     ;;
    esac
    
    if [ -n "$via" ]; then
 ssh $extra -A -t "$via" \
            ssh -t "$dest" "tmux -2 -L netbook attach \
            || tmux -2 -L netbook"
    else
 ssh $extra -t "$dest" "tmux -2 -L netbook attach \
            || tmux -2 -L netbook"
    fi
    
    stty sane
    echo "Dropped, press Enter to reconnect."
    if read x; then
 echo "Reconnecting..."
    else
 # Something bad happened to our tty. We'd better exit.
 exit 1
    fi
done

So now I can connect to work-host1 from home via the gateway, shut the laptop, travel to work, open it again, hit Enter and reconnect directly to work-host1 without losing any state. What's not to like? Well, there's still some more things we can do.

ssh-and-tmux: part one

So, I finally got round to buying myself a netbook. Unlike my old laptop the netbook has a battery that actually works so I can pick it up, use it for a few minutes and then close the lid so it suspends. Later on I can pick it up again and carry on where I left off. At least that's the theory. The problem is that quite a lot of what I do ends up involving ssh connections to remote hosts and whilst those connections survive a brief period of suspension they don't last for too long.

I've used Screen on and off for over fifteen years and I thought that this would help to solve the problem. Not long after implementing the solution I switched to tmux.

Step one is to initiate tmux over the ssh connection:

ssh -t destination "tmux -2 -L netbook attach || tmux -2 -L netbook"

The -2 forces tmux to assume that the terminal supports 256 colours. All of the ones I use seem to.

The -L netbook option gives the tmux session a unique name so it won't get mixed up with a manually launched tmux session.

First I try and attach to an existing session but if that fails I create a new one.

I put this in a script named ssh-and-tmux.

Step two is to automatically reconnect when the connection is dropped. I decided not to do this automatically because that would keep throwing off other connections from elsewhere. Leaving multiple connections active at the same time would mean that the usable window size might be limited too. Instead I simply wait for the Enter key to be pressed before trying to connect again and if anything goes wrong exit.

So, the step two script is:

#!/bin/sh
if [ -n "$TMUX" ]; then
    echo Already in tmux
    exit 1
fi

if [ -n "$STY" ]; then
    echo Already in screen
    exit 1
fi

dest="$1"
while true; do
    ssh -t "$dest" "tmux -2 -L netbook attach || tmux -2 -L netbook"

    stty sane
    echo "Dropped, press Enter to reconnect."
    if read x; then
        echo "Reconnecting..."
    else
        # Something bad happened to our tty. We'd better exit.
        exit 1
    fi
done

Sometimes it can take ssh a while to notice that the connection has been dropped. In that case I can simply type " ~ . " to kill the ssh connection and reconnect.

This works well but there's more to come in the next post.