How to send multicast packets via a specfic interface in Linux
Having tried all possible ways couldn't find a work around for this problem. I have a machine with two interfaces eth0 and eth2. I want all ff38:40:2001:dead:beef:cafe::/96 packets to go on eth2. I tried all the following but when I do ping6 ff38:40:2001:dead:beef:cafe::1 the packets always goes on eth0. Things I have tried and have not worked (i.e., packet still goes out on eth0).
$> route add --inet6 ff38:40:2001:dead:beef:cafe::/96 gw 2003::100 dev eth2
$> route add --inet6 ff38:40:2001:dead:beef:cafe::/96 dev eth2
$> route add --inet6 ff38:40:2001:dead:beef:cafe::/96 metric 1 gw 2003::100 dev eth2
My routing table is
[root@dev ~]# route --inet6 |grep eth0
fe80::/64 * U 256 0 0 eth0
ff00::/8 * U 256 0 0 eth0
[root@dev ~]# route --inet6 |grep eth2
2003::/64 * U 256 68 0 eth2
fe80::/64 * U 256 0 0 eth2
ff38:40:2001:dead:beef:cafe::/96 2003::100 UG 1 0 0 eth2
*/0 fe80::c671:feff:fe14:e482 UGDA 1024 0 0 eth2
ff00::/8 * U 256 0 0 eth2
However, ping6 ff38:40:2001:dead:beef:cafe::1 -I eth2 work just fine. Moreover, I see this problem only on Linux machines (MAC is fine).
[root@dev ~]# ping6 ff38:40:2001:dead:beef:cafe::1 -I eth2
PING ff38:40:2001:dead:beef:cafe::1(ff38:40:2001:dead:beef:cafe:0:1) from cal eth2: 56 data bytes
64 bytes from 2012::1: icmp_seq=0 ttl=253 time=19.1 ms
64 bytes from 2012::1: icmp_seq=1 ttl=253 time=2.16 ms
64 bytes from 2012::1: icmp_seq=2 ttl=253 time=2.14 ms
64 bytes from 2012::1: icmp_seq=3 ttl=253 time=2.26 ms
64 bytes from 2012::1: icmp_seq=4 ttl=253 time=2.08 ms
64 bytes from 2012::1: icmp_seq=5 ttl=253 time=2.15 ms
root@dev ~]# uname -a
Linux 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
Perhaps the problem has to do with the fact that there is a ff00::/8 for eth0. How do I overrule that route. I am not able to delete ff00::/8 route as well.
Answer
I'm not entirely convinced my solution is correct, but I can at least shed a little more light on what is going on.
Background
Linux actually has multiple routing tables, and they are searched one at a time in a specific priority order until a table with a matching route is found. You can optionally search some of the routing tables based on source address or protocol; see the ip-rule(8) man page.
The trouble is the "local" routing table, which has priority 0, the highest possible. The "local" table is populated automatically by the kernel and holds "obvious" interface and broadcast routes. For IPv6 under Linux, this apparently includes the entire multicast block.
The Problem
I'm going to be using the iproute2 tool rather than the more traditional route, because it will show me everything I need to know.
On my Linux box:
$ ip -6 route show table local
local ::1 via :: dev lo proto none metric 0
local fe80::213:a9ff:fe91:5bcb via :: dev lo proto none metric 0
local fe80::250:b6ff:fe44:37d1 via :: dev lo proto none metric 0
ff00::/8 dev eth0 metric 256
ff00::/8 dev eth1 metric 256
$ ip -6 route show table main
fe80::/64 dev eth0 proto kernel metric 256
fe80::/64 dev eth1 proto kernel metric 256
ff15::/16 dev eth1 metric 1024
ff00::/8 dev eth1 metric 1024
$ ip -6 rule show
0: from all lookup local
32766: from all lookup main
...And my multicast packets for ff15::1 (5==site-local, >link-local) end up on eth0, because the "local" routing table matches first and overrides the "main" table, even though the "main" table has a more specific route. This overriding behavior is correct in the greater scheme of policy routing, but the choice of auto-adding ff00::/8 to the local table is questionable to me.
My Solution
I don't have enough experience to know if this is a good idea, but:
# ip -6 route add ff15::/16 dev eth1 table local
and now my ff15::1 packets are routed through eth1.
This agrees somewhat with the semantics of the local table, in that it's routed directly through a device. It doesn't feel exactly right (considering automatic management and "you shouldn't have to look at this table"), but it's the best solution I've found.