Sun NICS and pfSense 2.1.x (FreeBSD 8.x)

Opening Ramble

I’m doing more and more networking these days. I’ve got some Juniper hardware on order but I’m also futzing around with pfSense running on some old thin clients (builtin crypto hardware FTW!!). Because I don’t have copious free time, I ordered some prebuilt pfSense boxes from ebay. $70 and 4 days later I had a marginally well built box more than suitable for replacing any OpenWRT or DD-WRT box in my place, I have many.

Why get rid of *WRT?

First, I’m not getting rid of all of my *WRT boxes. I’m simply repurposing them in ways that make them less critical to my infrastructure. That said I’m doing so for a number of reasons, any of which would be sufficient grounds IMNSHO for dumping them altogether. 

1. iptables — I’ve been managing firewalls since 1998 and my blood pressure rises every time I have to deal with that POC masquerading as a firewall solution.
2. community & support — pfSense is actively maintained but I don’t actually care about that because I know how it’s built. I could build my own starting with NanoBSD. DD-WRT has packages out there for supported routers that haven’t been updated i many years. OpenWRT went through several bouts of inactivity and rapid development. I could learn the tool chain for getting the WRTs built and deployed or I could just use a system I already know and support myself if it ever comes to that
3. hardware — The *WRTs run on a wide variety of cheap hardware that’s readily available.  Much of that hardware doesn’t have builtin crypto acceleration and isn’t very expandable (if at all). 
4. commitment & knowledge — The pfSense guys own their project and it’s issues. While they work to keep things in line with FreeBSD they don’t abdicate responsibility for what they ship. They take it upon themselves to patch in new features, retrofit, and push changes upstream.

This really isn’t a comparison of one project versus another. There are places where one may be appropriate and not another. I’m not bashing the *WRTs. I just prefer to have a little linux running on my network as possible. D’oh!!! did I type that out loud?

The arrival

The boxes I’d ordered came with quad-port PCI Sun NICS (hme[0-3]) for a total of 5 ports including the builtin Ethernet port (vr0).  They were pre-installed with pfSense 2.1.4 configured for VGI/DVI output, vr0 set as WAN and hme0 set to LAN. 

I don’t know why it is that people buy used computing devices and use them without doing a full reinstallation to make sure that they have a clean base to start with. When I’d done a new installation I noticed that my Sun NICS were behaving oddly. They’d start up and be “connected” then after just a few seconds they’d show “unconnected”. The link light would intermittently show solid green light then go dark. Turns out this is a known issue and has a pretty simple fix.

If you can get to a shell on the device you simply need to set the media and mediaops flags like so: 

ifconfig hme0 media 100baseTX mediaopt full-duplex

Do this for each interface that’s flapping and you should be good.

Status…so far

So far, things are going smoothly. I’m not seeing any issues that were typical with the *WRT boxes I had. It’s only been two days so who knows? I may do an update after I’ve had time to gather some meaningful data. 

side notes:

My boxes are NeoWare CA22/E140. They include the Via C7 with PadLock Security Engine. These things tend to get warm. That’s the case even when run for their intended purpose.  

How to make a (real) network bridge with a Mac (mini) and OS X (Mavricks)

This is a quick post about setting up a simple layer 2 network bridge using a Mac (in this case a 2011 Mac Mini) running OS X (Mavricks). My goal was to setup the Mac, henceforth referred to as the mini, as an access point for my main network. I found no resources on line that covered doing this so I decided to write it up for those are interested.

What we’re not doing

I found many posts online that covered setting up “Internet Sharing” with a Mac. Some of those referred to this as ‘bridging’ which it is not. At least, in it’s simplest form it is little more than Network Address Translation (NAT) with packets being forwarded from one interface to another.  The problem with that approach is that only IP Traffic is passed and even that is adulterated such that clients on one side (the inside) of the link are not directly addressable by hosts on the other side (the outside) of the link.

It’s great for letting multiple machines browse the web but not good for having multiple machines  talk to each other whether they’re plugged in at the switch or connecting over Wifi and pulling from a common DHCP pool of addresses.

Configure Primary Interface

Primary is conceptual here. I’m taking about the interface that I’ll use to configure the host from over the network. The IP address for this interface will be the (main) address for this host. For me the primary interface is the ethernet port on the back of the Mini. Yours could be any of USB, BlueTooth, USB, FireWire, etc.

We want to make sure that the interface is properly setup on the network. For you that may mean DHCP configuration, manually IP-ing it,  or something else. I’m not going to spend time telling you how to do this. You’ll know it’s working when you can ping something external to the box (preferably external to the network).

In my case I set a static lease in my DHCP server so that every time the mac address for the Mini shows up it gets the same IP. The interface happens to be en0.

Reboot and make sure it works on start up.

Turn on Internet Sharing

Yes, I know what I said before. We’re using this to make sure that we can actually connect to the WIFI access point and get packets out on the network.

Go to System Preferences -> Sharing

On the left-hand side under Service click on Internet Sharing but make sure you do not click the ‘on’ checkbox next to it.

Share your connection from: should be set to Ethernet (at least for me).

To computers using: should have (only) Wi-Fi selected (the checkbox to the left). 

Click on  Wi-Fi Options and setup the SSID, password, and channel you plan to use in the end. You can use test values if you want, just remember to set them appropriately before you’re done with the final step.

When you’re sure everything is setup as you desire, click the checkbox next to Internet Sharing to enable it and click OK or Start if prompted by a subsequent dialog box to (re)start Internet Sharing.

Now configure a wireless client to connect to your Mac’s wireless network. Try to ping something to make sure it works.

Reboot and make sure it works on startup.

Bridge the interfaces

Internet Sharing is going to interfere with binding the two interfaces together into a bridge.  So the first thing we have to do is open a terminal window and type:

sudo launchctl unload -w\

/System/Library/LaunchDaemons/com.apple.InternetSharing.plist

That disables Internet Sharing. The actual command to create the bridge is remarkably simple. At the command line type in: 

sudo ifconfig bridge create

The output of this command depends on what other bridge devices may exist already.  If you get something like bridgeX where X is a number then that is your new bridge device and  you now only have to add interfaces to it:

sudo ifconfig bridge0 addm en0 addm en1 up

For me en0 is my ethernet port, en1 is the Wi-Fi device, and bridge0 is the name of the device returned by the previous command. Your device names/numbers may be different. Use ifconfig to help figure it out. 

Once you’re done with the above you need to re-enable Internet Sharing: 

sudo launchctl load -w\
/System/Library/LaunchDaemons/com.apple.InternetSharing.plist

Connect your devices and test to make sure that you can get a proper LAN IP address.

Making it last

You’ll want to run the command line steps every time the system starts. I do this with a cronjob for the root user.  I store it in ~root/bin/ (/var/root/bin) and invoke it with the line:

@reboot /var/root/bin/bridge.sh


The script contains:

#!/bin/sh -x 
#Name:      bridge.sh
#Purpose:   To create a layer 2 bridge between two interfaces allowing Ethernet
#           frames to pass between them. 
PATH=/sbin:/bin

if1=${1:="en0"}  # If no primary interface is passed assume en0
if2=${2:="en1"}  # if no secondary interface is passed assume en1

ifconfig >> /tmp/ifconfig.out
echo "#############################################################################"
runas_root(){
    #Helper funciton to insert "sudo" if succeeding command is invoked a non-root user
    case ${UID} in
        0) echo "" ;;
        *) echo "sudo " ;;
    esac
}

$(runas_root) launchctl unload -w /System/Library/LaunchDaemons/com.apple.InternetSharing.plist
bridge_if=$( $(runas_root) ifconfig bridge create)
$(runas_root) ifconfig ${bridge_if} addm ${if1} addm ${if2} up
$(runas_root) launchctl load -w /System/Library/LaunchDaemons/com.apple.InternetSharing.plist
echo "#############################################################################"
ifconfig >> /tmp/ifconfig.out

Reboot and make sure it all works on start up

Happy hacking...