Pressing Alt+F4 to switch to the installer output, I could see Attempting to install GRUB to a partitionless disk or to a partition. This is a BAD idea.

It would appear that my USB disk was detected as /dev/sda, and my target hard disk was detected as /dev/sdb, based on the output from cat /proc/partitions.

So I fixed it by activating a shell in the Alt+F2 session and ran the following:

chroot /target grub-install /dev/sdb

Then I switched back to the installer (Alt+F1) and hit “Continue without bootloader”.

Once installation had finished (I unplugged the USB stick), I was presented with a grub> prompt, so I followed the instructions here, as follows (this assumes that your boot disk is /dev/sda:

grub> ls
(hd0) (hd0,msdos5) (hd0,msdos1)
grub> set root=(hd0,msdos1)
grub> ls /
lost+found/ etc/ ...
grub> ls /boot
config-3.11.0-26-generic ...
grub> linux /boot/vmlinuz-   # press Tab
                             # then add the following
          root=/dev/sda1
grub> initrd /boot/initrd    # press Tab
grub> boot    

Once the system is booted, log in and then:

sudo update-grub
sudo grub-install /dev/sda