Campus Network Design

My role in modernizing our work network started a little over a year ago when I joined the network team.  Now to better understand our network, it’s best to think of it as a campus network.  We have a lot of buildings, with only a few having a large amount of users, connected all together in a switched network.  In total, I believe I have over 100 switches right now currently making up our network.  The list of issues on our network are many:

• All users, switch management, servers, printers, etc. all on VLAN 1
• Access switches daisy chained together
• Switches handling routing that are not the core switches of the network

Now outside of everything on VLAN 1, some may ask what exactly is wrong with this layout.  Well for one, it’s very difficult to locate devices on the network, since all devices are on the same subnet.  Thankfully, we have a Fluke Optiview that can locate devices in a large layer 2 network.  The network is also prone to flooding, from broadcasts, unicast, and multicast flooding.  In fact for a couple of weeks, we experienced unicast flooding due to a misconfiguration between our core switches and the newly-installed-but-completely-out-of-my-control layer 3 switches. (Maybe one day I’ll do a post about outsourcing and it’s “benefits”).

So I’ve set out to improve the network.  I am working to model the network after the Cisco Campus Network Design.  The basis of this design is that access switches, routers, VPN routers, datacenters, etc. connect to distribution switches, and the distribution switches connect to two core switches.  The two core switches are the control center for the network.  Now, if you follow the Cisco recommendations, the Core switches only handle layer 3, and the distribution switches handle layer 2 and layer 3.  Recently, there was a design discussing bringing routing all the way down the access level.

I see two issues with bringing routing all the way to the access layer in our network.  First, we have a guest VLAN for our contractors that is spread through one vlan in our network, and we have a security VLAN for our security network that is spread through one vlan in our network.  VRF-lite is basically VLANs for layer 3, but I believe each network in the vrf would be in it’s own subnet.  However,  I haven’t verified this yet.  If this is the case, then that will increase management.  Second, with over 100 switches, we are talking about 200 separate networks (data and voice) on each switch.  That is 200 subnets to manage dhcp address for compared to the 2 we have now.  This would be a huge increase in management.  Granted, we could easily locate devices on the network by just performing a traceroute.  Broadcast flooding or unicast flooding would be limited to only one switch.  With the access switches doing routing, you could standardize on a similar configuration for all switches.

But do the increased efficiencies in the network offset the increased management of the network?

Lets forget about routing at the access layer, and go back to the design with the distribution switches being layer 2/3 devices.  I would still have to look at using VRF-lite for the security and guest networks.  With about 8 distribution spots in the network, I could have a minimum 16 vlans (1 voice and 1 data vlan for each distribution point) and 16 subnets, but you would have more than one access switch per vlan.  A limit could be set and have say 5 access switches per vlan.  This would increase the number of vlans and subnets to manage, but would reduce the number of devices in each vlan.  But in this setup, there is still an issue of locating devices on the network.  If one went with the minimum setup of 16 vlans and 16 subnets, private vlans could be used to isolate traffic between buildings, but even private vlans are not without their security issues, so this would need to be taken into account.

All in all, it appears I have a daunting task ahead of me.  At some point, I will have to make concessions to both manageability and efficiency in the network.  Though this exercise has made me think about subnets and manageability vs efficiency.  Say I wanted a different subnet for each access switch.  I install a 24 port switch.  Is it really efficient to assign a /24 subnet to this switch just for management sake since this is what most people are used to?

I’ll update this with links once I figure out what’s going on with not being able to add links.  I believe it may something to do with using Safari 4 Beta.

Advertisement