Disabling reverse-path filtering in complex networks

2009-07-13 21:25 +00:00 | 9 comments

I recently created a very complex network using routers running Ubuntu Hardy. These routers were configured with the following features:

failover shared IP addresses using heartbeat
routing announcements via Quagga BGPd
802.1q VLAN tagging
multiple physical interfaces

During debugging of this network, I encountered an odd scenario whereby traffic coming in from the external interface (eth0) could not reach the IP address of the secondary (inactive) router’s internal interface (eth1, VLAN tagged).

dual-routers

When router B is inactive, it is available on IP addresses on both eth0 and one VLAN of eth1. Because it is inactive, traffic bound for an IP on router B eth1 moves in a big clock-wise circle around the above diagram. It comes from outside to router A eth0, exits router A via eth1, moves to the internal switch, then back to router B eth1. When active, traffic comes directly to it from outside, and instead router A behaves this way. The problem is that when in the inactive state, the inactive router’s return traffic goes directly out eth0. That is, it comes in eth1 and goes back out eth0.

When such a situation occurs (source and destination traffic to the same IP using different interfaces), the Linux kernel (by default in Debian/Ubuntu) drops the traffic as potentially spoofed. This is called reverse-path filtering, and it is useful for preventing obviously bad traffic from entering your network (sourced from a local IP address, multicast, etc). However, you can safely disable it if you perform the same sanity checks in your iptables firewall (and you do, right?).

To disable this check, use the net.ipv4.conf.all.arp_filter sysctl variable:

root@routera:~# sysctl -a | grep \\.rp_filter

net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.eth1.rp_filter = 1
net.ipv4.conf.eth1/10.rp_filter = 1
net.ipv4.conf.eth1/20.rp_filter = 1

In this example, my router has two physical interfaces, eth0 and eth1, and has two tagged VLANs on eth1 (VLANs 10 and 20). All are set to perform basic reverse-path filtering. Change them by editing /etc/sysctl.conf. Find this section:

# Comment the next two lines to disable Spoof protection (reverse-path filter)
# Turn on Source Address Verification in all interfaces to
# prevent some spoofing attacks
net.ipv4.conf.default.rp_filter=1
net.ipv4.conf.all.rp_filter=1

Comment-out the two lines here, and apply changes with sysctl -p. If any interfaces still show rp_filter=1, reboot or set them to zero yourself:

sysctl -w net.ipv4.conf.eth0.rp_filter=0

This problem with rp_filter seems to specifically cause issues with VLAN interfaces. I have another pair of routers doing the same thing, except they don’t use VLAN tagging on eth1. They have no issues with rp_filter at all.

If you suspect your traffic is being filtered by rp_filter, you can find out with log_martians:

sysctl -w net.ipv4.conf.all.log_martians=1

Your syslog will now show dropped packets.

Note that the value of rp_filter is not what you may expect.

Tags: kernel, networking

9 comments

Comments feed for this article

Trackback link: https://www.tolaris.com/2009/07/13/disabling-reverse-path-filtering-in-complex-networks/trackback/

Pingback from Multicast Packets Being Dropped/Blocked on 2012-08-12 at 21:20 +00:00
iMartin on 2012-11-05 at 02:11 +00:00

Thanks !! It worked for me.

I had 2 network interfaces (eth0 and eth1) with 2 public IPs. The traffic from outside where always routed to eth0 by default … but nothing for eth1. This article allowed me to make it work the way i wanted.

Reply
Matt on 2013-01-24 at 19:19 +00:00

Worked for me too thanks a bunch!

Reply
Pingback from Linux VLAN Bridge - Just just easy answers on 2013-09-06 at 03:50 +00:00
Pingback from OpenVPN client without redirect-gateway doing triangular routing and IP spoofing not working on Ubuntu - Just just easy answers on 2013-09-07 at 02:02 +00:00
Mike on 2014-01-05 at 03:02 +00:00

This just resolved an issue that was bugging the hell out of me!
Any interface above eth0 suddenly stopped working on my kvm vm’s, everything looked perfectly correct on the HV and the vm but the interfaces couldn’t send any packets out over the non primary ip addresses. Changed to net.ipv4.conf.eth1.rp_filter = 0 and it instantly started working.

Thanks a lot for the info!

Reply
Mike on 2014-01-05 at 03:14 +00:00

As an update after looking into the implications of this change you may wish to also try 2 instead of 0 (2 Worked for me on the secondary interfaces). Info on the settings from the RHEL documentation below.

rp_filter – INTEGER
0 – No source validation.
1 – Strict mode as defined in RFC3704 Strict Reverse Path
Each incoming packet is tested against the FIB and if the interface
is not the best reverse path the packet check will fail.
By default failed packets are discarded.
2 – Loose mode as defined in RFC3704 Loose Reverse Path
Each incoming packet’s source address is also tested against the FIB
and if the source address is not reachable via any interface
the packet check will fail.

Reply
1. Tyler Wagner on 2014-01-08 at 17:28 +00:00
  
  I don’t think “2” is of any use here. If your server has a default route, then the behaviour of 0 and 2 will be the same.
  
  Reply
2. A Yasir on 2017-08-17 at 22:32 +00:00
  
  Very useful article rp_filter is definitely an issue with a lot of routing issues. The log_martians sysctl parameter is a great debug tool. I wish Linux had an integrated TCP debug tool or log though!
  
  Cheers
  
  Reply