The Good, the Bad, and the Ugly

... or "trolling for pics on Flickr"

I was poking around on Flickr looking for shots of other people's data centers and wiring closets. For instance, you can look for all pictures tagged with "wiringcloset"

And you can find things like a picture looking upwards at a rack of 1U boxes, which I found in the l0calh0st group

Some other groups with pictures of other people's network/computer/cabling gear:

• secret life of technology
• electronic porn
• rackmonkeys
• data center of shame

That last group is where I found these shots of a data rack that got rewired: before and after
Searching for "cabling" gets you everything from knitting to data centers. Using tag clusters to narrow the search down to more wire-geeky (as opposed to fiber-geeky) gets you something more like this.

and I can't help but agree with the caption on this picture: "No matter what runs on top of the communications infrastructure, at the bottom will always be something like this—a bunch of wires connected together somewhere out of sight."

yeah.

(nb: I'm taking a couple of weeks off from posting for the holidays. I may go through and tag/retag a bunch of posts. Apologies if this freaks out rss feeds or osmething)

I never knew you could get tcpdump files from an ASA

I never knew you could get tcpdump-format files from a cisco ASA. Very cool!

The capture process is the same, but then stop the capture with the command "no capture $CAPNAME interface $INTERFACE" instead of "no capture $CAPNAME". Then you can go to the ASA's web site and find the file in "http://$FIREWALL/capture/$CAPNAME/pcap"

Download it and then you're good to open it with tcpdump or wireshark or whatever your packet capture viewer of choice is.

It doesn't look like there's a way to tftp it rather than turning on the ASA's http server for the duration of the download.

Passwords and security

Public Service Announcement and discussion!
Gawker media's database of email addresses and passwords for several hundred thousand users was copied and distributed and password crackers are eagerly attempting to crack everything there and see what else uses those same ID's and passwords.

Password reuse is a common exploit vector -- It seems convenient to use the same low-security password for multiple web forums and other things. But it really does open up holes that lead to bigger exploits. (See also this xkcd.) In particular, don't re-use a password for an email account or anything connected to your credit cards or banking. Using a base password and then some added characters for each web site can be a convenient way to have different passwords that are still reasonably easy to remember. And then there's always PasswordSafe (my favorite), which runs on several common platforms, to securely store passwords.

A bit of whimsey in the middle of my switch

Just a quick amusing thing, today.

If you open up a cisco 3548 switch and take a look at the circuit board, there's a Buddha:

Click on the thumbnail to follow the link to bigger versions of the picture.

Radia Perlman did a Google talk and it's on Youtube

"Routing without tears; Bridging without danger".

Because there's no crying in routing!

802.1x peeve

So, I'm trying to set up 802.1x, using MSFT's IAS server to do authentication against windows domain accounts. Seems straightforward enough.

I've set the switch up, which was pretty simple. The IAS server is being difficult, however. I've got it set to not send any extra VSA (vendor-specific attributes), but it's sending several along with the authentication approval that seem to make the switch choke. I've got a ticket open with MSFT about it.

It's not clear to me what's going on, here. I'll post again when I figure out more.

Cisco IOS CLI vs Foundry/Brocade CLI

The CLI for foundry and cisco devices are very similar. Some commands are identical:
show interface
show run
enable

Others are slightly different:

cisco             foundry
-------------     ----------------
show ip int br    show int br

description       port-name

speed 100  
duplex full       speed-duplex 100-full

Some are notably different, such as putting a given port into a given VLAN (natively, not with 802.1q tagging or anything):
cisco:

vlan 100
int Fa 0/15
switchport mode access
switchport access vlan 100

foundry:

vlan 100 name MySpecialVLAN by port
untagged eth 15

gender and drupal

Interesting post over at Wild Unicorn Herd

Open Source people, and Drupal people in particular, pride themselves on having a “doacracy”—a community that values getting stuff done above traditional authority. This could create a beginner-friendly, non-hierarchical environment of subversion and experimentation. In practice we just have white straight cis men getting SUPER DEFENSIVE at the suggestion that maybe they got where they are not only by the sweat of their brow, and shouting down any mention of patriarchy, racism, or any other systemic oppression when people run the numbers and get to wondering why there’s so little minority representation in Open Source.

On the other hand, I'm pleased to see things like the discussion over the expanded 'gender' field at Drupal.org (link here).

hat tip to geekfeminism.org

more than a thousand times better than the nexus 6! take that, roy batty.

Cisco's hot new multiservices platform is the Nexus 7000. I haven't actually laid hands on one, but I figured I'd read up on them a bit. Thus, I blog about it.

It's a move away from IOS-based platforms -- it runs a newfangled "NX-OS". It's a big-assed chassis switch that runs either 10 or 18 modules; each of those configurations takes two supervisor modules, like the old supervisor cards on the Catalyst switches. The bigger chassis can take up to 516 gigabit ethernet connections which is some pretty beefy port density, right there.

The OS, NX-OS, has a similar CLI as IOS to keep with the cisco look and feel, but it is apparently a completely different beast than IOS under the hood: it's built on an embedded Linux kernel. (Historical trivia: IOS is somewhat derived from TENEX, the nonstandard OS for the PDP-10.) Wikipedia tells me that NX-OS doesn't support per-user logins or scp, that it considers all access-lists as extended (about time -- I think IOS's way of handling acl's dates back to when people were running multiprotocol networks that handled more than just IP), and that it doesn't support the 'write' command and instead uses the 'copy' command, which I can live with. It's derived from SAN-OS, which I am also unfamiliar with.

I believe the technical term for this device is "ginormous". The small one, which has 10 horizontal cards, is about half a rack high:


And for scale, here's another one of the 10 slot models next to its 18 slot (arranged horizontally) big sister:


Looking at it, I'd say that even with the cards and power supplies and fans stripped out, you still need a 19" wide lift that can slide inside your cabinet to install it.

Cheerleading. FOR SCIENCE.

So, there are plenty of instances of women - and men, too - in tech, engineering, science, etc, doing things to show that smart people can be hot. (Sexy oncologists calendars, things like that). It's not quite as often that you see something showing that hot people can be smart. Science Cheerleaders are former (possibly a few current - not sure) professional cheerleaders who have found careers in science and engineering who go around and CHEERLEAD FOR SCIENCE.

Breaking down expectations and stereotypes for nerds and cheerleaders! This is fabulous. I think the only way this could be more awesome is if they were *really* some kind of Banzai Institute sort of group who went around the world doing goodwill appearances that were actually a cover for their real mission of SAVING THE WORLD.

Their web site is at www.sciencecheerleader.com, and here's a video of them at the USA Science and Engineering Festival.

Cisco PIX/ASA order of operations for NAT

From this page on cisco.com:

inside-to-outside:

* If IPSec, then check input access list
* Decryption—for Cisco Encryption Technology (CET) or IPSec
* Check input access list
* Check input rate limits
* Input accounting
* Policy routing
* Routing
* Redirect to Web cache
* NAT inside to outside (local to global translation)
* Crypto (check map and mark for encryption)
* Check output access list
* Inspect context-based access control (CBAC)
* TCP intercept
* Encryption

outside-to-inside:

* If IPSec, then check input access list
* Decryption—for CET or IPSec
* Check input access list
* Check input rate limits
* Input accounting
* NAT outside to inside (global to local translation)
* Policy routing
* Routing
* Redirect to Web cache
* Crypto (check map and mark for encryption)
* Check output access list
* Inspect CBAC
* TCP intercept
* Encryption

Another F5 iRule

I want to pull some percentage of the traffic off and send it to one web server pool and send everything else to a different pool.

Simplest way to do it would seem to be by the last digit of the number of seconds -- that gives me granularity of 10% at a time. To start off, I'm doing just 10% of the traffic, so I'll pick requests with seconds ending in "0"

when HTTP_REQUEST {
if { [HTTP::uri] equals "/directme" } {
set foobaz [clock seconds]
if { $foobaz ends_with "0"} {
pool special
}
else {
pool normal
}
}
}

A bit more about the F5

The F5 local traffic manager (LTM) is a load balancer. Here's a quick overview of how it does its basic thing:

Individual web servers are defined as "nodes". Each node can be part of one or more "pools" of web servers. These pools are then assigned as a resource to any of the "virtual servers" that you define.

When you set up a virtual server, you have to indicate what port and protocol you're using. If it's for SSL connectivity and the F5 is handling the encryption, you'll need to indicate what certificates you should be using.

You can specify what healthchecks it should use to determine if a node is up, and what metrics the F5 should use to figure out who to hand the next connection off to when it uses a pool containing several members.

Writing an F5 iRule to block traffic by user-agent

So, the F5's can do all kinds of swoopy things using the iRule scripting language. I've been playing around with simple ones.

Suppose I want to block all the traffic from a certain robot that advertises itself as having the user-agent field "AnnoyingRobot". I could use an iRule like this to block it:

when HTTP_REQUEST {
if { [HTTP::header "User-Agent"] contains "AnnoyingRobot" } {
drop
return }
}

The "contains" operator looks for a substring, so it'll catch "AnnoyingRobot/4.5" and "AnotherAnnoyingRobotButDifferent/mozilla".

The next step would be to have it search against a list of user-agents. The way to do this is with a "class" or "datagroup" (the terms appear to be interchangeable in F5-speak). So you can use the GUI to create a "string" type datagroup named "userAgentsToBlock" that contains:

AnnoyingRobot
RegisCustomUseragent
OtherStuffWeBlock

and then change the rule to be like this:

when HTTP_REQUEST {
if { [matchclass [HTTP::header "User-Agent"] contains $::userAgentsToBlock ] } {
drop
return }
}

Then apply that iRule to a given virtual server, and you're all set to drop traffic from user-agents you don't like.

SPOOOOOOOKY

So, my office (like many offices) has Halloween decorations up. Lots of cartoony graphics with headstones that read "RIP".

And I kept wandering by and reading "RIP" and thinking about the routing protocol.

So I made my own spooky Halloween decoration:

EIGRP

Except for the odd cisco class, I've never actually used EIGRP. It's another interior routing protocol, like OSPF and often used in place of it. It's cisco-only, as it's cisco's proprietary protocol, which makes it a non-starter in heterogeneous routing environments. (OSPF is based on standards and is supported by pretty much all vendors of "real" routers, which is to say "not a $30 home router for your dsl line".)

Unlike OSPF, EIGRP doesn't have the concept of "areas", so all of your internal network is lumped into a single monolithic administrative collective. Like OSPF, it establishes neighbor relationships with other routers and shares routes among its neighbors. EIGRP gives you a few more ways to tweak your routes, having bandwidth and delay instead of a simple cost metric and does route filtering a bit more granularly.

More about BGP -- load sharing across equal-cost paths

Suppose you have several equal-bandwidth or equal-cost paths between a pair of routers that want to be BGP neighbors. For example, two or more WAN links between two BGP areas. You want the routers to route over both paths. The thing to do is to use a loopback address for your BGP neighbors. You must also have "ebgp-multihop" configured as well, so BGP will be okay with going to a neighbor that is not on the same subnet as itself.

So, router A (AS 64591) and router B (AS 64592) are connected by a pair of WAN links.

On router A:

int loopback 0
ip address 10.255.255.1 255.255.255.255

router bgp 64591
neighbor 10.254.254.1 remote-as 64592
neighbor 10.254.254.1 update-source loopback 0
neighbor 10.254.254.1 ebgp-multihop


Router B looks similar:

int loopback 0
ip address 10.254.254.1 255.255.255.255

router bgp 64592
neighbor 10.255.255.1 remote-as 64591
neighbor 10.255.255.1 update-source loopback 0
neighbor 10.255.255.1 ebgp-multihop

Don't forget to include your loopback network range in the "network" statement for the BGP instance.

BGP basics: "Active" is not a good state.

BGP is TCP based. If you're got a router that wants to share BGP routes, each neighbor has to be explicitly configured. When this is done and BGP restarted, the router opens a BGP session to each of its neighbors on TCP/179 and chats about what routes it has and what routes it can learn.

If there are no routing updates, the routers exchange keep-alive packets every 60 seconds to be sure that the TCP session is still up and the routers are all happy.

When a BGP TCP session starts up, the BGP neighbor will show as state "Connect" when the router is actively trying to connect. If it successfully makes the connection, it shifts to state "OpenSent" and then through "OpenConfirm" to "Established".

If the router cannot make a successful TCP connection, it shifts to state "Active". NOTE: This does not mean that the connection has been made; just that it's actively trying to make the TCP connection. If your neighbors show as "Active", your routing is *not* working yet. Once the TCP session is established, the neighbor state goes through "OpenSent" and "OpenConfirm" to "Established". That last state, "Established" is the one you want to see in your summary of BGP neighbor states.

On a Cisco, your neighbor summary table would look something like this (taken from Cisco IOS IP Command Reference Vol 2 of 4: Routing Protocols, Release 12.3T - IP Routing Protocols Commands):

Router# show ip bgp summary 


BGP router identifier 172.16.1.1, local AS number 100 
BGP table version is 199, main routing table version 199 
37 network entries using 2850 bytes of memory 
59 path entries using 5713 bytes of memory 
18 BGP path attribute entries using 936 bytes of memory 
2 multipath network entries and 4 multipath paths 
10 BGP AS-PATH entries using 240 bytes of memory 
7 BGP community entries using 168 bytes of memory 
0 BGP route-map cache entries using 0 bytes of memory 
0 BGP filter-list cache entries using 0 bytes of memory 
36 received paths for inbound soft reconfiguration 
BGP using 34249 total bytes of memory 
Dampening enabled. 4 history paths, 0 dampened paths 
BGP activity 37/2849 prefixes, 60/1 paths, scan interval 15 secs 

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down State/PfxRcd
10.100.1.1      4   200      26      22      199    0    0 00:14:23 23
10.200.1.1      4   300      21      51      199    0    0 00:13:40 0

There are three fields that you're probably most interested in under normal circumstances:

• The first one to look at is "State/PfxRcd": If the TCP session is up and the routers are exchanging routes, then this field will be a numeric value that shows how many route prefixes have been received from the remote neighbor.
• The next one of note is "TblVer", which is the routing table version. Under normal circumstances, this should be the same if routing is stable.
• The third field of interest is the "Up/Down". If the BGP session to the neighbor is up and happy in the "Established" state, this is the duration of the current TCP session. If it's not up and happy, this will display the current state of the connection. Remember, "Active" is not what you want to see here. (Yes, I'm harping on it, but I've seen many people make the mistake of thinking that "Active" means that the connection is working correctly.)

This table (Table 41 in the same document) lists all the fields in the output and what they indicate.

Stubby and NOT SO STUBBY.

OSPF uses the concept of "area" to describe different chunks of the network.  The center of the network is "area 0", the backbone area, and it's assumed that all areas connect to area 0. 

Router interfaces, not entire routers are part of an area.  So a router can span more than one area -- for example, a router could have one interface in area 0 and another interface in area 1 and a third in area 199.  Other routers in area 1 would then see the router as a path back to area 0. 

Areas can be defined as "stubby", meaning that there's only one path out from the area, and so external routes don't get advertised to the stubby area.  A "not so stubby area" (nssa) can receive intra-area routes, but no external routes.

Who's the boss? (OSPF edition)

So, a router interface that's participating in OSPF floods the network with a HELO packet -- this is sent to a multicast address, so if you've got acl's in play make sure they allow the OSPF multicast.  One of the things the HELO traffic does, besides just announcing "hey, I'm here" to any likely devices that might also be looking for neighbors, is share out the information used for election of the "Designated Router" (DR) and "Backup Designated Router" (BDR). 

The DR and BDR serve as points of contact for exchange of routing information -- instead of each router updated every other OSPF neighbor with link state announcements, all the devices update the DR and BDR and they send out link state updates to all of the OSPF neighbors.  This reduces the complexity of the exchange of routing information.

DR and BDR election is done by OSPF priority.  If two interfaces on a given network segment have the same priority, the higher Router ID is used as a tie-breaker.  For any given OSPF device, the "Router ID" is the highest IP address on the box (including loopback interfaces).  Specific router interfaces may have an OSPF priority set explicitly as well to adjust whether or not they become DR; a priority of 0 means that the interface should never be the DR or BDR; this is described as the state "DROTHER"

Dress for success....

Apparently, when dealing with at least some VC's, "dress for success" means "wear a white penis":
'white male tech startups get funding for being white and male'. 

Also, the Wall Street Journal has an article looking at reasons that women-owned companies are generally smaller than men-owned companies (in spite of there being more new women owned businesses than men owned businesses for decades).  (And Discover magazine points out that women become more risk-averse when faced with stereotypes of women as risk averse)

more vacation. content later.

Regis is still on vacation.  Actual blog posts back on Weds.

No post - vacation!

Regis is in vacation and is not writing a long post on her phone.

OSPF types of routes

If you look at an OSPF routing table ("sh ip route ospf", for instance), there are a couple of different types of routes that might be displayed.

• "O" -- An OSPF route from within the same OSPF area.
• "IA" -- An OSPF route from a different OSPF area.
• "E1" -- An OSPF route that is a Type 1 external route.
• "E2" -- an OSPF route that is a Type 2 external route.

An External route is one that is learned from another routing protocol and redistributed into OSPF.  Type E1 is an external route that includes the path metrics from the redistribution other routing protocol and the cost to get to the router that's doing the redistributing.  Type E2 is an external route that only includes the cost from the redistributed routing protocol and does not pay attention to the cost to get to the border router that is doing the redistributing.

Open Most Complicated Routing Protocol First

OSPF! Open Shortest Path First.  It's an interior gateway routing protocol - you'd use OSPF within a given AS, for instance.  It has a whole bunch of things that can be tweaked.  Most people (in my experience) use it as their interior routing protocol.

OSPF is based around the idea of a link state diagram -- that is, a diagram of the network connections between all the routers in a given OSPF instance.  When making routing decisions, a given router will figure out what the best ("shortest") next-hop is for a given destination.  If left untweaked, the route metric will be based on the number of hops and the bandwidth of each link (lower-bandwidth links are less preferred).  You can adjust that and manually weight routes to encourage traffic to go over a path that it would otherwise not prefer.

An OSPF-enabled router uses broadcast packets to announce itself and to find out about other OSPF routers in the same collision domain.  If compatible, the routers will establish a neighbor relationship, and exchange route information.  Among neighbors, there's a designated router and a designated backup router.  The designated router is a given router interface that is in charge of sending out link state announcements and letting the other router interfaces know what's up with the links.  It's a specific interface, not a specific individual router -- a router that is a DR on interface 1 might not be the DR for a separate OSPF instance on interface 2.

Useful tool

Wireshark -- tagline "sniffing the glue that holds the Internet together -- is a network sniffer that uses tcpdump format capture files.  It runs on windows and OSX.  Very useful, with fairly versatile filtering available.  What I particularly like, though, is the ability to follow and display a given TCP session.  It's great for tracking a given web session or something like that.

BGP decisionmaking, and a link to the bad science blog about blind prejudice

or "Fucking BGP - how does it work?" context

I'm on a BGP kick because someone was asking me questions about BGP last week and I felt like an idiot when I couldn't remember more than two of the criteria BGP uses to make its routing decisions.

So, here's how BGP makes its routing decisions (and the order in which it evaluates criteria):

1. Pick the route with the highest "weight" (bgp "weight" is a cisco-specifc thing specified on the local router)
2. Pick the route with the highest "local pref"
3. Prefer a route that is locally originated vs remote.
4. Pick the route with the shortest AS path (You can avoid this by using the "bgp bestpath as-path ignore" command.)
5. Pick the route with the lowest origin type. (Interior routing protocols such as OSPF are preferred over routes that originate via EBGP)
6. Pick the route with the lowest "MED", Multi Exit Discriminator.
7. Prefer EBGP over IBGP.
8. Prefer the path with the closest (determined by interior routing protocols) next hop.
9. The router determines if anything MultiPath-related needs to happen.
10. If we've gotten to this point and still have multiple choices, prefer the oldest route. This helps prevent route flapping.
11. Prefer the route from the peer with the lowest Router ID. This is sort of like how ospf picks a designated router by "router ID". For BGP, the router ID is the highest IP address on the router, preferably loopback addresses. You can avoid IP address tweaking by manually setting the router ID with the command "bgp router-is".
12. If you're running an environment with route reflector(s), prefer the shortest cluster list among multiple paths that go to the same originator/router-id.
13. If not, just pick the path with the lowest neighbor IP address.

Then, RELEASE THE PACKET! (It helps to imagine the TUBEZ of the internet being full of tiny little kraken. Or maybe that's just me.)

...
Meanwhile, over on the unsurprising gender side of things:
On the Internet, no one knows you're a dog. But if you're a woman, what you wear can change people's perception of your skills, at least for musicians.

bgp

bgp! very important!
here's a bgp overview from apricot 2004, and here's cisco's big page of BGP resources

More detail in later posts.

An elementary link

yeah, who forgot to click "publish" yesterday? YES, IT WAS ME!

If you're looking to go back to basics, or you're just learning basics, cisco's "Internetworking Technology Handbook" is a pretty good place to start.

It's available online in HTML or for download as a bunch of pdf's.

no post, holiday

holiday weekend. no post. go have fun.

cue the doctor evil voice

Soon, we will see ethernet switches that run of speed of ONE HUNDRED GIG.

Which, you know, that's pretty sexy right there.

Soon, I'm sure, you'll be able to buy 10G switches in blisterpacks at the checkout aisle at microcenter.

A couple of interesting links

It's a few months old, but Sarah Mei's post about why she doesn't work at Google puts her finger on one of the differentiators between success and failure: smart people vs smart people who get shit done.

As I get older, I'm becoming more aware of the explicit and implicit ageism in tech.

This techcrunch article saying women in tech should stop blaming the men because everyone loves the women pretty much assumes its conclusion from the get-go:

Success in Silicon Valley, most would agree, is more merit driven than almost any other place in the world. It doesn’t matter how old you are, what sex you are, what politics you support or what color you are. If your idea rocks and you can execute, you can change the world and/or get really, stinking rich.

Update on the SYN-less ACK problem

So, a while back, I mentioned a weird one I'd been seeing -- SYNless ACKing.

Here's what we knew:
It only happened on the first connection.
And it only happened after the machine has been idle for long periods of time for a given host.

It turns out that it's a known issue. According to Sun Oracle, it's part of CR 6942436.

on the less technical more feminist side of things

Mentoring is a good thing. I'd missed this report about mentoring at Sun Computer. It includes some mentoring best practices (and some "what not to do"), among other things.

And for a frivolous bit of feminism, I find the twitter feed from FEMINIST HULK often amusing and occasionally charming. Samples: "HULK FIND COMPLICITY BETWEEN ALL SYSTEMS OF OPPRESSION. RESULT: HULK HAVE VERY DIVERSE PORTFOLIO OF SMASH." and "BIG GREEN FISTS NOT ONLY WAY TO SMASH PATRIARCHY. BUT HULK REALLY LIKE CONFETTI-EFFECT OF PRETTY HEGEMONIC DEBRIS."

OMGCOOL Map of undersea cables.

This is very cool.

http://www.cablemap.info/, a map of undersea cables.

I wish they'd put the split somewhere on a continent so the Pacific Ocean was a single map section. Ah well.

NOC NOC.

A few links about NOC-building.
Ringel/Brown presentation from NANOG 24
TERENA NOC group. The ITIL Visible Ops handbook, which is more process than NOC, but still pretty good.

Unrelatedly, a businessy article about gender in the workplace and how not to do it.

Vacation and one-to-one NAT on a Juniper Netscreen

Sorry - forgot to mention I was going on vacation. Mmmm. Sweet vacation.

Anyways, back to the nerdy salt mines.

There are a couple of ways to do address translation on a Juniper Netscreen firewall. The simplest is a "MIP", or Mapped IP. This creates a one-to-one NAT that can work in either direction, and no port translation is done. You define a MIP on a specific interface that handles the subnet the NAT is on (the ingress interface).

So if you're setting up a NAT where folks on the Untrust network can talk to a host on the Trust network using a NAT address in the Untrust zone, you'd set it up on the interface in the Untrust zone.

From the GUI, it's Network > Interfaces > Edit > MIP (List). From the CLI, assuming this is all in the "trust-vr" virtual router, the command looks like this:

set interface "ethernetX/X" mip {nat address} host {real internal address} netmask 255.255.255.255 vr "trust-vr"

eg, assuming my outside NAT address is 10.1.1.100 on interface 3/2 and my internal address is 172.168.1.100:

set interface "ethernet3/2" mip 10.1.1.100 host 192.168.1.100 netmask 255.255.255.255 vr "trust-vr"


When making policies via the GUI, the Untrust zone will include "MIP(10.1.1.100)", so you can just select that as destination and the firewall will do the right thing for inbound traffic. For outbound traffic, select the internal real address as the source, and the Netscreen will do the right thing and NAT the traffic as it leaves the outgoing interface.

sniffing traffic on a netscreen - snoop or debug flow

If you want to figure out what's happening with traffic on a Juniper Netscreen, there are three basic ways to do it. The least informative is to turn on the "log" option for a given rule. This provides a gui way to look at packets that have matched the rule.

The most informative is "debug flow", from "debug flow basic" up to "debug flow all", and you'll get all kind of information about the decisionmaking process the netscreen goes through as part of passing or dropping a packet - is it part of a flow, is it a new connection, how does it get routed, etc.

And in between, as a happy medium, is the snoop command. It gives you output similar to the basic output from the Solaris "snoop" command. Adjusting the "detail" level will let you change the length of the packet it examines.

Both debug flow and snoop let you filter what traffic is interesting by means of the "set ff" command, and both drop their output into the "dbuf" debug buffer.

"Wait, did we forget something?"

It's not just overt displays of stupidity and classlessness on the part of the industry. Sometimes it's more subtle. Like making a management simulator on the web and FORGETTING TO INCLUDE THE OPTION TO HIRE WOMEN.

I can't really thing of anything to say beyond what what's here at contexts.org (via geekfeminism.org)

*sigh*

Password recovery on foundry/brocade devices

As usual, you need console access to a brocade/foundry device that you want to do password recovery on.

Power cycle the device, and when prompted, hit the "b" key during the boot process. Once you're in boot monitor mode, you can enter "no password" to tell it to (you guessed it) ignore the password data. Then, tell the device to boot by issuing the "boot system flash primary" or "boot system flash secondary" command, as appropriate. Once it boots, you can set the password to a known value.

Everything is better with Legos.

It's true! Everything is better with Legos. Check this out: The folks over at Righteous IT have illustrated the TCP/IP packet header structure with Legos. Aww yeah.
https://righteousit.wordpress.com/2010/06/27/practical-visual-three-dimensional-pedagogy-for-internet-protocol-packet-header-control-fields/

It's Radia Perlman's network. We just route over it.

I dropped a couple of routers into place, replacing a bunch of network-specific firewall interfaces with a backbone-type link and put several of the specific networks onto the routers. Basically, pushing the firewalls deeper into the core, and the routing further out to the edge.

NOTE TO MYSELF: It is all Radia Perlman's fault, which is to say -- spanning tree spanning tree spanning tree. Check it, then check it again and still again. There were several problems I ran into that were either caused by spanning tree configs being borked or that I diagnosed by looking at spanning tree. If any of your network devices on a vlan disagree as to what the root bridge ID is, then you have some kind of discontinuous vlan. If you have spanning tree on some but not all devices, you are going to run into all kinds of problems with loop topology.

Oh, blade chassis devices, you are awesome yet annoying.

So, a couple of different folks are making blade chassis devices. IBM has the bladecenter and HP has the bladesystem.

While they're awesome in terms of machine density, they can unexpectedly hose you if you're not careful. For instance, if you don't carefully check that there are no unexpected internal paths that bypass spanning tree, you could end up with a topology loop.

I hate when that happens.

Stay classy, network world.

Oh, yay! Network world has a list of resources and pages regarding women in networking.

Oh, boo! Network world is still running a hottest booth babe contest!

I am reminded of a Cisco Networkers conference I went to about 10 years ago. One of the sessions was about ways to encourage and retain women in networking. That was nice. And then, the very same day, there was the social event where part of the decor added to the venue was a bunch of "living statues", which were scantily clad women standing around on pedestals being very still. No guys. Just women. Even a single token male bit of beefcake to go with the cheesecake would've been great and would've gotten the taste of boyzone sexism out of my brain. NO SUCH LUCK.

STAY CLASSY, NETWORK INDUSTRY. STAY CLASSY.

Okay, this is a weird one -- Synless Acking

So, the normal TCP three way handshake looks like this:

Client       Server
SYN --------->
   <---------  SYN/ACK
ACK --------->

I'm seeing packet traces that show my first connection attempt does this:

Client       Server
SYN --------->
   <---------  ACK (with no SYN)
then nothing until a new SYN from the client

Which ... I mean ... That's a big pile of WTF going on there.

I'm not quite sure what's going on, here, but *something* is.

expect!

Do you need to do a bunch of things to network devices? This only works if you have a stable password instead of some kind of two factor authentication like an RSA token, but if that describes your circumstance, then check out expect, available at http://www.nist.gov/mel/msid/expect.cfm. It's a scripting language for automating stuff. So, for instance, you can write a script to log into routers, modify access lists, change passwords, whatever.

I first used it back in the mid 1990's when I worked in a place that had a training room. The access lists on the routers were different depending on whether the training class was external users or internal users, and we didn't want the external users being able to get to the internal network resources. The solution was to have an expect script that fired off at the end of every day and force the router to have the more secure access list. That way, if the acls had been changed during the day to allow for external users, it would be sure to be set back the way we wanted it.

also, this is my 25th real non-lorem-ipsum blog post. yay!

vacation

Regis is on vacation this weekend, so no blog post.

NAC!

How NAC works. Because I am lazy and it is late, I am cribbing from a paper I wrote earlier this week:

The core functionality of NAC involves three parts: the supplicant, the authenticator, and some kind of authentication server. The supplicant is an edge host device that wants to access network resources over the local LAN. The authenticator is the network device -- usually a network switch -- that can provide that access if and only if the supplicant is correctly authenticated. The authenticator requests some kind of authentication credentials from the supplicant to determine what action to take based on the response from the authentication server. The authentication server is, unsurprisingly, a server that validates the credentials provided and informs the authenticator whether the supplicant is or is not allowed to access the LAN; they authentication server may also provide additional information to the authenticator about the user after a successful authentication. Until the negotiation is complete and a supplicant is correctly authenticated by the authenticator, the supplicant has no access to the the actual data network on the switch. Physical layer connectivity is provided, but an unauthenticated port is not part of any LAN or VLAN.

A switch acting as an authenticator will operate as a normal switch until a live network host is connected to a port configured for 802.1x. The link state change triggers the 802.1x authentication process. In normal non-802.1x operation, the switch would add the newly live switchport to the correct VLAN. For an 802.1x port, the switch is not added to a VLAN but instead placed in an unauthenticated state, which allows only 802.1x protocol traffic. The authenticator switch sends initiator datagrams that serve to request credentials. A correctly configured supplicant machine will have an operating system that is configured to listen for these packets and collect the authentication credentials from the user. They are often some kind of user/password authentication pair linked with the specific user of the supplicant machine, but may include certificates associated with the machine itself rather than the user.

The supplicant host machine and the authenticator network device exchange authentication credential information. This will include the kind of authentication, based around the Extensible Authentication Protocol originally developed for the PPP authentication, as well as the authentication credintials themselves. Communication from the authenticator switch and the authentication server is usually handled via the RADIUS authentication protocol. RADIUS allows for additional information to be exchanged besides a simple yes/no verification, so that configuration data specific to the user or a user group can be communicated back to the authenticator. The authentication server evaluates the credentials and replies to the authenticator switch to affirm or deny the access. If the authentiation server affirms the credentials, the authenticator switch will place the port onto a pre-configured VLAN or network segment. At that point, the supplicant machine can continue normal procedures for connection to the network and issue DHCP or BOOTP packets if needed, or begin normal connectivity if its network interface is already correctly configured.

Links on a Sunday morning.

A couple of interesting links from around:
the geek feminist blog looks at 'doing girl stuff' in foss.

theage.com.au has an article about how women managers are perceived differently. Color me unsurprised.

And this is an interesting quote that caught my attention:
You have to bring your expertise to a place where it’s magical, and show them stuff that’s bleeding edge to them, but normal to us. As Arthur C. Clarke said: “Any sufficiently advanced technology is indistinguishable from magic.” What you do isn’t magic in your circle, so you have to go somewhere where it is. (from http://inoveryourhead.net/how-to-get-paid-for-what-you-do-for-free/)

faster than pigeons

I've been poking around to see who is still using FTP on a network. The tcpdump command is

bash-3.00$ sudo tcpdump -i nge1 -nS 'tcp[13] & 2 != 0' and 'port ftp'

FTP can be a bit twitchy to troubleshoot. It's an old protocol -- dating back to 1971 and RFC 114.

Originally, it was pretty straightforward: there are two ports in use, the control port (tcp/21) and the data port (tcp/20). The client connects to the control port and they have a lovely chat until time comes for the actual file to transfer. Then, the server opens a connection *from* its tcp/20 port to a port specified by the remote client. Which is why firewalls freak it out -- it involved an inbound connection.

PASV mode, or 'passive ftp', avoids some of this by letting both sides negotiate a high (above 1023) port, the server opens that port for connection and the client makes the connection to that port.

All of which makes it a fracking pain to troubleshoot from a network point of view.

A couple of useful OIDs that I keep forgetting

This is more a "notes to myself" entry, but here are the couple of OIDs I keep misplacing:

get the list of operational vlans on the switch
snmpwalk -c commstring MySwitch 1.3.6.1.4.1.9.9.46.1.3.1.1.2

get the list of mac addresses the switch knows about per VLAN
snmpwalk -c comstring MySwitch@VLAN .1.3.6.1.2.1.17.4.3.1.1
for whatever VLAN is. So, it would be "MySwitch@177" for vlan 177.

Interface names for each port (not descriptions/port-names, names)
snmpwalk -c commstring MySwitch .1.3.6.1.2.1.31.1.1.1.1
IF-MIB::ifName.2 = STRING: Gi1/1
IF-MIB::ifName.3 = STRING: Gi1/2

While Regis is off writing a paper, here are a couple of links...

I'm in the midst of writing a paper, so it's an extra-short blog post today, but I did want to keep to my schedule.

Here's a couple of interesting links:
A presentation from NANOG 40 about extra-high bandwidth network stuff and the pros and cons of 40G and 100G technology.

and brocade has some crazy cloud network switch fabric going on. I'm curious if (or maybe just when) this kind of tech will trickle down to a more moderately-sized enterprise level rather than very large scale data center type networks.

minimal post.

If you haven't read Kirrily Robert's excellent keynote speech from OSCON, you might want to give it a read or watch it on blip.tv. She takes a look at barriers to women in open source projects.

I'm interested to see what's worked for encouraging more women to get involved in open source development. I think the dreamwidth idea for having a "#dw-kindergarten" IRC channel set up to help newbies without snarking them all to hell and back. I like the idea of bypassing the "you must be this tall to ride this ride" kind of mentality. I know I've certainly been nervous about asking for help from some communities because of the inevitable trolls who snark at folks. So, it's good to see successful alternatives.

The Internet is changing

Turn and face the strain ... ch-ch-changes

I read this back shortly after it went up after NANOG 47, and I came across the URL again the other day. It's a fascinating look at the shift in the Internet behavior over the past couple of years. Two of the big shifts are the transition from money coming from connectivity to money coming from content (inc advertising), and the huge spike in cable internet due to things like the comcast "triple play" deal

A few things that stuck with me about it:
In 2007, thousands of ASNs contributed 50% of content.
In 2009, 150 ASNs contribute 50% of all Internet traffic.

In 2009, Comcast is now a net contributor of Internet traffic.

The PDF is at http://nanog.org/meetings/nanog47/presentations/Monday/Labovitz_ObserveReport_N47_Mon.pdf

Demystifying 10G modules

I don't currently have any 10G gear in networks I manage, but I've been occasionally looking at 10G products so that when I need them, I'll have a clue about what the heck is going on with that market segment.

Looking through old NANOG presentations, I found a pdf from a presentation about 10G fiber pluggable modules. http://www.nanog.org/meetings/nanog42/presentations/pluggables.pdf

This gives a great overview of what's out there and where the market is apparently going in terms of hardware.

Some handy commands for the F5 LTM

Some useful commands:

• To show data on existing connections, use "b conn show" for a condensed list, or "b conn show all" for more information about the connections.
• "bigpipe interface 1.1 media show" to show the speed/duplex of an interface - in this case, int 1.1
• To add a static route for traffic to use the management network, "bigpipe mgmt route <destination network> netmask <netmask> gateway <management gateway>"[1]
• To set a remote syslog device, "bigpipe syslog remote server 1.2.3.4"

And an important bit of data: unlike most every other damned piece of gear I deal with, the console connection for the F5 LTM is 19200 baud, not 9600 baud.


[1] For example, if your NTP server should talk to the management interface instead of one of the functional interfaces

Return to basics.

So, I'm starting a graduate program. It's a bit nerve-wracking since my college career was a bit ... sporadic. But the first class I'm taking is "Data Communications."

I do not expect the bulk of the subject matter to be particularly challenging. :) I'm mostly going for this as a first class to ease myself back into the routine of actually doing school work and all of that.

However, I think it will be interesting to go back to the very basic principles. It doesn't look like this will be a class where they hand me a copy of the Comer book and say "Here is some hex. Decode the packet," but still, it should be interesting to go back to looking slowly at things my brain normally glosses over and take a look at Internet connectivity with a bit of beginner's mind.

ssh and sexism

WTTWFTVOE (Words To The Wise From The Voice Of Experience): when attempting to set up SSH on a network device (switch, router, etc), it works a WHOLE HECK OF A LOT BETTER when you remember to actually have a domain name configured.

On a different topic, I find myself fascinated by this article from theage.com.au, which is an excerpt from The Hidden Brain. It takes a look at a couple of instances of sex bias, from people's reactions to near-identical descriptions of a person that only vary in the gender of the person described, to comparing the experiences of two transgendered academics at Stanford - one MtF and one FtM.

I think I've been fairly fortunate in avoiding some of the high levels of sexism that other women I know have experienced in the working world. Perhaps some of it is just a comparison of IT with my previous career in the construction trades. And some of it is having ended up working with and for good folks. I've certainly encountered a fair bit, but mostly it's been at a distance as far as direct "sorry, can I talk to a real engineer" and "are you the admin?" level of stuff.

On the other hand, there's this great post from Kate fuckin' Harding about women tooting their own horn and getting past that cultural expectation for women to be demure about their abilities.

Quickie useful link

Just a quick link: a page with undocumented cisco commands: http://www.elemental.net/~lf/undoc

nyt: why so few women in silicon valley

By virtue of being the kind of network engineer I am, I'm unlikely to ever be a single-digit employee of a startup. By the time a company needs a full-time employee to handle the kind of infrastructure that I do, they're usually well into the scores of employees. And I haven't really done much of the startup thing. The smallest company I worked for was about 50 people, and even that was fairly well established.

I spent the entire late 1990's tech boom working for a single very large company. It was great fun, and I got to work on big networks and big projects. But it means that there is an entire subculture of the tech industry that I haven't really directly experienced very much -- that kind of technical nomadic thing that I've seen a bunch of friends in the SF bay and other areas go through, moving from startup to startup. (It also is one of the reasons that I've never been laid off, which feels like it makes me a huge outlier in the IT field!)

But even having avoided dealing directly with venture capital firms, I'm kind of appalled by this article in the NYT relaying some experiences an ex-HP manager type had pitching her company idea to VCs.

she recalls one venture capitalist telling her that it didn’t matter that she didn’t have business cards, because all they would say was “Mom.”

Seriously? It's the 21st century! Hey boys, 1975 called, and they want their chauvinism back. Also, 1975 can keep those horrible not-a-scarf-not-a-tie thing women wore with business blouses, too. We don't need those. Hates them, my precious, hates them we do.

troubleshooting

Here's a problem I ran into. I set up a new environment with a different layer 3 infrastructure. And everything seemed to be working. Until I got reports that some people couldn't get to one network in the new environment.

So, traffic going to the range 10.2.84.0/24 was working great for some people, but for others, it was not -- TCP connections would connect and then fail after a few seconds.

I spent a bunch of time looking at switch ports, and spanning tree, looking to see where the blocking ports were and making sure that there's no loops. So I gave up on that line of inquiry and started tracing back at layer 3.

Sure enough, there were two routes for 10.2.184.0/24 in one of my core routers - one of which pointed to the right place, and one of which pointed to the wrong place.

I deleted the wrong route, and things worked again.

Using tcpdump to only capture SYN and FIN packets

Sometimes with a network capture, all you want to know is when a session starts and when it finishes. So you don't actually want to capture anything beyond the session start and finish handshakes. Here's how to do it:

tcpdump -w flagdump 'tcp[tcpflags] & (tcp-syn|tcp-fin) != 0'

Low Estrogen Zone

I've been working for the past few days in a big data center.

I know that I'm not actually the only woman in the building, because I've seen a woman on the janitorial staff wandering around.

100M ethernet over copper/RJ45. I hate it.

As near as I can tell, way too many networking vendors handle autonegotation incorrectly for 100M ethernet.

I mean, it could be worse -- things could fall back to 10M.

If a fastethernet port is set to autonegotiate, when it sees link, it tries to - as you might guess - negotiate the speed and duplex. If the device at the far side doesn't negotiate in a compatible fashion or at all, a port set to autonegotiate will fall back to half duplex.

If this happens, then at best you end up with both sides at half duplex (annoying but not catastophic) or with a duplex mismatch. A duplex mismatch can totally hose your performance.

Honestly, when dealing with 100M, I'd just as soon hard-set everything to full duplex and be done with it.

Chicks in IT

I'm a bit busy, so this is just a bunch of links that I have found interesting as a woman working in the IT field.

Back in 1991, Ellen Spertus wrote an excellent piece "Why Are There So Few Women in Computer Science". Some things have changed since then, but not as much as one might hope.

Some linux folks carried on with How to Encourage Women in Linux

A good bit on reddit from "TwoXChromosomes" with some information snippets about women in computing and other fields.

An LA Times article "Men Who Explain Things. Ahh. Male Answer Syndrome.

"No, I thought *you* were supposed to keep track of the password!"

"Would your holiness care to change her password?" --Hackers

Password recovery on the cisco 2900 series is a little bit more involved than password recovery on a 2800 -- you need physical access to the switch to push a button. Remote control power outlets won't help you, here. Instead of typing the "break" sequence at boot time, you need to push the "MODE" button.



So, yeah, connect up your console, power cycle the router (there's no power switch, so you have to unplug/replug it), and the press and hold the "MODE" button as you power it up.

The top LED above the MODE button is labelled "SYST". This LED will begin to flash green during the POST, and - if you are holding down the "MODE" button -- will eventually turn solid green. At that point, you can release the "MODE" button.

You should see console output that looks something like this:

Base ethernet MAC Address: de:ad:be:ef:ca:fe
Xmodem file SYSTem is available.
The password-recovery mechanism is enabled.
Initializing Flash...

Base ethernet MAC Address: 00:1a:a1:44:f5:00
Xmodem file SYSTem is available.
The password-recovery mechanism is enabled.

The SYSTem has been interrupted prior to initializing the
flash fileSYSTem.  The following commands will initialize
the flash fileSYSTem, and finish loading the operating 
SYSTem software:

    flash_init
    load_helper
    boot


switch: 

Next, issue the "flash_init" command so it notices its flash memory and the "load_helper" command:

switch: flash_init
Initializing Flash...
flashfs[0]: 602 files, 19 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 32514048
flashfs[0]: Bytes used: 12889088
flashfs[0]: Bytes available: 19624960
flashfs[0]: flashfs fsck took 10 seconds.
...done Initializing Flash.
Boot Sector FileSYSTem (bs) installed, fsid: 3
Setting console baud rate to 9600...

switch: load_helper

switch: 

If you do a directory listing of the flash drive, you should see a file named "config.text", which contains the startup configuration for your device. Renaming that to something other than "config.text" (or, if you want to do a full wipe, deleting it) will cause the switch to boot up without a startup config.

switch: rename flash:config.text flash:config.old
switch:

Issuing the "boot" command will start the rest of the boot process. Once its up and ready, answer "no" to the initial configuration prompt.

switch: boot
Loading "flash:c2960-lanbasek9-mz.122-25.SEE4.bin"...@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@
File "flash:c2960-lanbasek9-mz.122-25.SEE4.bin" uncompressed and installed, entry point: 0x3000
executing...

              Restricted Rights Legend
[blahblahblah]
Switch   Ports  MODEl              SW Version              SW Image            
------   -----  -----              ----------              ----------          
*    1   26     WS-C2960-24TT-L    12.2(25)SEE4            C2960-LANBASEK9-M   




Press RETURN to get started!


Would you like to enter the initial configuration dialog? [yes/no]: no
Switch>

Since the switch doesn't have a config, you can go into enable MODE without needing to use a password. Then you can load the old config into running memory.

Switch>ena
Switch#copy flash:config.old running
Destination filename [running-config]? 
Failed to generate persistent self-signed certificate.
3298 bytes copied in 0.361 secs (9136 bytes/sec)
My-Pet-Switch#

And now you can change the password to whatever you want.

My-Pet-Switch#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
My-Pet-Switch(config)#ena sec Y0ullNeverGuess
My-Pet-Switch(config)#line con 0
My-Pet-Switch(config)#pass ThisIsMyPW
My-Pet-Switch(config)#line vty 0 15
My-Pet-Switch(config)#pass ThisIsMyPW
My-Pet-Switch(config)#
My-Pet-Switch(config)#^Z
My-Pet-Switch(config)#wr mem

Don't forget to save the config to memory!

If you want to keep the old config for some archival purpose, you can tftp it off. Otherwise, you can delete it with the "delete flash:config.old" command.

"Wait, I thought you knew the password..."

"My voice is my passport. Verify me."

So, there you are, with a cisco 2800 series router that you need to reconfigure. And you're all set to take some down time, reconfigure it, and drop back into service.

Only you don't have the password for it. The password is lost, or forgotten, or was typo'd initially. Something like that. Which makes any further configuration tasks kind of difficult.

Password recovery on the cisco 2800 is pretty trivial. You don't even have to do a full wipe to factory defaults, so no configuration data is lost. You need to be able to power cycle the router, so you need to have either physical access or the ability to remotely control whatever outlet is feeding your router. If you've got that, then it just takes a console connection and about 10 minutes of downtime. Non-enable exec access to the unprivileged command line interface is helpful, but not required.

First, make sure you know how to send a "break" signal with whatever you're using to connect to the console - whether directly to the console from a serial port on your computer, or via some kind of remote console server.

Connect to your router's console port. If you've got the CLI access, run the command "sh ver | inc register". Your output should be something like:

Configuration register is 0x2102

Make a note of that octal number. If you don't have CLI access, just assume it's 0x2102, because that's almost certainly what it is.

This is the configuration register, which tells the router important things at boot time, like "Should I load IOS? Should I load the config that's in my NVRAM?" What you need to do is tell it that, in fact, it should not load the NVRAM config on boot. This is done by modifying the config register. To do that, you need to get into "rommon" mode.

Power cycle your router, and then send the "break" sequence within the first 60 seconds of power-on.

It should display a message "System received an abort due to break key", and then a bit more text, and then the "rommon 1>" prompt. If it starts to self-decompress its IOS image, you've missed your opportunity and you'll need to power cycle the router and try sending the break sequence again.

Change the value for the configuration register with the command

confreg 0x2142

Reset the router with the "reset" command. At this point, the router should go through it's normal boot sequence, decompressing the IOS image and starting up IOS. But since it's not loading the configuration from NVRAM, it will ask you if you want to go through the initial configuration dialog, just as if it were a new router out of the box. You don't want the guided setup, so you can answer "n" to the question:

 --- System Configuration Dialog ---

Would you like to enter the initial configuration dialog? [yes/no]: n

Hit return a few times to get to the "Router>" prompt, and then go into enable mode -- you will not be prompted for a password -- and then tell the router to load the "startup-config" configuration from NVRAM into running memory.

Router>enable
Router#copy startup-config running-config
Destination filename [running-config]?

You should then see messages about interfaces coming up and other normal messages as the router processes your config information. Once it's done with that, you should have a prompt for the configured hostname of your router. At this point, you can change the passwords and enable secrets as normal, enable your live interfaces, and then save the config to NVRAM to save your changes.

MyPetRouter# conf t
Enter configuration commands, one per line. End with CNTL/Z.
MyPetRouter(config)# (stuff to bring your interfaces up)
MyPetRouter(config)#ena sec Y0ullNeverGuess
MyPetRouter(config)#line con 0
MyPetRouter(config)#pass ThisIsMyPW
MyPetRouter(config)#line vty 0 15
MyPetRouter(config)#pass ThisIsMyPW
MyPetRouter(config)#^Z
09:22:00: %SYS-5-CONFIG_I: Configured from console by console
MyPetRouter#wr t
Building configuration...
[OK]

Take a look at the configuration, and the interfaces, and verify that the router looks like you want it to -- that interfaces are up and happy. You don't want to be accidentally saving a config that has all your interfaces in "shutdown" mode. Since "no shut" is the default state, the config you loaded won't include the "no shut" commands explicitly. So make sure all your active interfaces are up.

Now, you need to reset the configuration register to the previous value so that the next time the router reboots, it will read the NVRAM config.

MyPetRouter(config)#config-register 0x2102
MyPetRouter(config)#^Z
09:29:00: %SYS-5-CONFIG_I: Configured from console by console
MyPetRouter#

Do a "sh ver" to make sure the configuration register reads "0x2102". If you have the spare time in your downtime window and you want to be extra sure, do a clean reload on the router and make sure it comes up happily. Because nothing sucks like being locked out of a router because of something you did in an attempt to not be locked out of a router.

A few links - pinouts and pictures

AllPinouts has a wiki full of pinout data: http://www.allpinouts.org/

Pictures of other people's stuff!
Via techrepublic... Server room cabling nightmares

On the other side, A company in Sweden has a computing environment fit for a James Bond Villain.

Big data centers! A list from datacenterknowledge.com

And also from datacenterknowledge, articles about the growing trend of Data Centers in a box.

Basics: LABEL YOUR EQUIPMENT...

The Kandinsky is painted on both sides! -- "Six Degrees of Separation"

Don't just label your gear, label it on both the front and the back. (Or the sides/top/bottom -- whatever sides people are going to be using to identify the equipment and work on it.)

This sounds kind of obvious, but I've worked in places where this seemed to be a strange new concept.

Clear labeling of your network equipment, on both the front and the back can speed troubleshooting - particularly if you've got someone doing the hands-on work who is less familiar with the setup. If you're likely to have colo staff doing remote hands work, be even more fastidious about your labeling.

Make sure that the label is visible. If your machine has a separate faceplate that needs to be removed for maintenance access, label the faceplate and label the surface under the faceplate. That way, if someone puts the wrong faceplate back on the machine, a person doing maintenance can see the mismatch.

Make sure that the label is well-secured. If it's falling off or has lost its adhesive, replace it or tape over it with clear adhesive tape to re-secure it.

I wrote an article for MSFT TechNet magazine a few years ago about DNS

DNS. Love it, hate it, avoid it with extensive hosts files... whatever. We're stuck with it.

I wrote an article about it a few years ago for MSFT TechNet magazine.
How IT Works: Domain Name System from the Winter 2005 issue.