Proxy ARP with Linux

www.internetsolver.com | Other Linux Documents

By David Weis with Internet Solver, LLC

AKA: firewalling a subnet with a single IP address

Why and How

Proxy ARP basically means that a particular machine (such as a firewall) will respond to ARP requests for hosts other than itself. This can be used to make a firewall mostly disappear from the machines on a network.

For an example, say you have a /28 subnet from your ISP that is routed through a Cisco router. Your router appears at the IP of x.x.x.97 with a network address of x.x.x.96 and a broadcast address of x.x.x.111. This leaves a usable chunk of 14 addresses for your hosts.

If you wanted to firewall these hosts from the internet without using proxy arp, you would need to either subnet your addresses and lose two more addresses for the new network and broadcast, plus half of your remaining IP's would be in the non-firewalled half.

Another method would be to have the firewall do port forwarding between all of the addresses to non-routed IP's (192.168.x.x) for your servers. Done properly, this would be okay. It isn't as transparent and may break some protocols like active FTP unless the firewall will compensate.

By using Proxy ARP, you can set up your machines in a DMZ to separate them from your client machines. This is also the least invasive method to set up, since you can keep the same IP's on all of the servers as you had when things weren't firewalled.

To set up the network, you will need a machine with at least two NIC's, three if you want to also masquerade client machines for outgoing access. Some variations on how I configured this are surely possible, but this is how I know how to do it and know that it does work. You will need a 2.4 series kernel, the iproute2 utility, and a recent iptables userspace program.

Doing it

You will need to set up the machine with the software mentioned above. Be sure to compile netfilter into the kernel by selecting yes for "Network Packet Filtering" under the "Networking Options" section. All of the pieces associated with netfilter are listed under "Netfilter Configuration" further down the list. It is probably easier to build each of the options into the kernel than use modules, there should be +- 22 choices to turn on.

Build your kernel, install it, and boot to make sure it functions. You should see some lines like

ip_conntrack (2046 buckets, 16368 max)
ip_tables: (c)2000 Netfilter core team
in the kernel boot messages (use dmesg if they went by too fast).

After you have your kernel running, build and install iproute2 and iptables. Instructions for doing so are in the packages themselves. Your distribution may have included them, but they are probably older ones. Grab the newest ones to make sure you have the right versions. Test them by running ip and iptables and see if they print something. Don't continue until they do.

After you have the above steps done, you will need to configure your network cards. This step should be done off of the network since you may end up with some conflicting addresses. Give two NIC's identical IP addresses, subnet masks, and gateways. The IP you choose needs to be an unused address on your network. In my case, I used x.x.x.98, since my router is at x.x.x.97. You could actually use about any address on the wire that isn't in use.

There is an example configuration available for download below that uses three NIC's, one for an internal 192.168.x.x network for client machines. If you want to create the file yourself, here are the steps. First off, enable Proxy ARP in the kernel. We'll assume your cards are eth0 and eth1.

echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/eth1/proxy_arp
Next, you will tell the kernel how to get to each of the two networks. When you are done, there will be a crossover cable from one NIC on your firewall to your router and the other NIC will be connected to the port on your hub/switch that the router was previously plugged in to. The kernel now thinks that the same machines are on each wire, which definitely won't work. We'll set it straight with these commands:
ip route del x.x.x.96/28 dev eth0
ip route del x.x.x.96/28 dev eth1
ip route add x.x.x.97 dev eth0
ip route add x.x.x.96/28 dev eth1
What we've done is first say that no network is reachable via either NIC, then say that the router (x.x.x.97) is connected to eth0 and the rest of your servers are hooked to eth1. I would suggest using this setup (router on eth0, servers on eth1) since the downloadable example assumes that.

We'll now use the ip command to verify our work. Running ip addr should yield something like this:

1: lo:  mtu 3904 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
2: eth0:  mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:90:27:3f:23:9c brd ff:ff:ff:ff:ff:ff
    inet x.x.x.98/28 brd x.x.x.111 scope global eth0
3: eth1:  mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:90:27:3f:4d:6c brd ff:ff:ff:ff:ff:ff
    inet x.x.x.98/28 brd x.x.x.111 scope global eth1
and ip route should show this
x.x.x.98 dev eth0  scope link 
x.x.x.98 dev eth1  scope link 
x.x.x.97 dev eth0  scope link 
x.x.x.96/28 dev eth1  scope link 
127.0.0.0/8 dev lo  scope link 
default via x.x.x.97 dev eth0 
After the routing tables and IP addresses look okay, you can turn on IP forwarding in the kernel by doing this:
echo 1 > /proc/sys/net/ipv4/ip_forward
At this point, you could take your new firewall, hook up the ethernet cables correctly, and have identical functionality (hopefully) to what you have now with no firewalling. Of course, this machine doesn't really do any firewalling, grab the IP tables howto from the link below and set up your tables. You can also use the example script provided below.

Troubleshooting

First, verify that you can ping all of the machines from the firewall, including the router. The main problem you can run into is that the router will probably cache the ARP entries for the machines for quite a while. The easiest way to solve that is to kill the power to your router and plug it back in. Try to ping again.

Let me know if you have any problems, there is a link to my email address at the bottom of the page, along with a place that you can post questions directly on this page.

Downloads

Other Sources of Information

Other Stuff

This is a reply to a question on comp.os.linux.networking where I first explained how to do some of this. Above is a cleaned up response. It's here as a source for my copy and paste. :-)

What you want is called proxy-arp. You would set the two NICS to both
have the same real IP address, such as .55 in your second diagram. Then
you do this
echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/eth1/proxy_arp

You will need to use the 2.4 kernel and get iproute2, then try these
commands

ip route del x.x.x.32/27 dev eth0
ip route del x.x.x.32/27 dev eth1
ip route add x.x.x.33 dev eth0
ip route add x.x.x.32/27 dev eth1

This tells the kernel that the router is the only thing reachable via
eth0 and the rest of the network is on eth1 (change as needed). This
will make the linux machine respond for the machines on either side.
Finally, do this

echo 1 > /proc/sys/net/ipv4/ip_forward

to enable routing. Either wait a while for the arp caches to expire or
restart your router. At that point you should be able to get back and
forth between the router and the other servers on the network. If you
look at your arp cache on a server, it will show the mac address of the
router as the mac address of eth1 on your linux firewall.

After you have this layer working, you can add your rules.

Good luck!
dave

"Norman D. Megill" wrote:
>
> I want to set up an IP filter in the following setup.  Right now every
> machine is connected directly to the internet with static (real) IP's in
> subnet X.Y.Z.32/27.  X.Y.Z.35-X.Y.Z.54 are DHCP'd with an NT server and
> the others are hard-configured in various machines.  There is currently
> no firewalling.
>
> Current setup:
>                                                          internet
>    other X.Y.Z.32/27 ------------------------------------ISP feed
>      NT/W98 machines      |                            X.Y.Z.32/27
>                           |                            gw=X.Y.Z.33
>                           |
>                           |
>                     NT web server
>                       X.Y.Z.60
>
> The management of the Windows-only office is more paranoid about Linux
> than about security, and furthermore wants no work disruption or risk
> caused by changing IP setups on various machines.  To demo an initial
> firewall, it must be "transparent" so that if it has a problem the
> internal network cable can be instantly unplugged from Linux and hooked
> back to the ISP feed to restore the current configuration.  My idea is
> that once we get it working we can slowly start to tighten things up,
> move the machines to masq'd/forwarded 192.168.x.x's, etc.
>
> Without getting into the security problems that will be initially
> present with this "firewall", if I don't get a "transparent" mode to
> work there is not going to be any firewall at all, or at best it might
> be an NT machine for management comfort.  The current setup has been in
> place for a few years with no apparent security problems, and "if it
> ain't broke don't fix it".
>
> Because both sides of the firewall are on the same subnet, I have not
> been able to get it to work after experimenting with all kinds of
> routing and ip setups.  I am starting to look at bridging, which may be
> the only solution, but the fact that it (I think) sets the NICs to
> promiscuous mode seems against the spirit of the firewall.  But before I
> give up I'll ask about it here.
>
> I am using RedHat 7.0 with kernel 2.2.16 and RH security updates.
>
> Experiment 1:
>                                   eth1       eth0
>    other X.Y.Z.32/27 ----------------- Linux -----------ISP feed
>      NT/W98 machines      |     X.Y.Z.55   X.Y.Z.56    X.Y.Z.32/27
>                           |                            gw=X.Y.Z.33
>                           |
>                           |
>                     NT web server
>                       X.Y.Z.60
>
> Experiment #1 doesn't work because a packet from the internet to
> the X.Y.Z.60 server makes the ISP gateway think that X.Y.Z.60 is
> on its local cable, and it sits there doing "arp who-has" for X.Y.Z.60.
> Of course Linux never responds because it only looks for packets
> to X.Y.Z.56.
>
> Experiment 2:  Same as Experiment 1 but added X.Y.Z.33 to eth1:0 and
> added X.Y.Z.60 to eth0:0
>
> Experiment #2 allows Linux to see the gateway arp's, but it seems to
> think the packets are for itself and I have found no way to
> transparently forward them to the eth1 side.
>
> Now, it seems that what I want should be theoretical possible, but I
> can't figure out how to make Linux can do it (without promiscuous
> bridging).  Adding to my puzzlement is that Figure 3-2 on
>
>    http://www.bb-zone.com/FWHowTo/chapter3.html
>
> shows the exact setup I want.  The author seems to say that the
> FW_ROUTER variable just bypasses the spoofing filter, but that would not
> solve the problem (since for now I have ipchains completely open).  He
> references "the SuSE firewall script," but there is no FW_ROUTER
> variable in SuSE's firewals-2.6-33.rpm if that's the script he means.
>
> Thanks for any advice.
>
> --Norm


djweis@internetsolver.com