This is just a simple cheat sheet I'll likely expand over time to capture all the useful information pertaining to fibre optic communications.
Connectors
The most common data connectors are outlined below
LC
Lucent Connector
Typically used for 4Gbps+ Fibre Channel and 10Gbps Ethernet communications. Most structured cabling within a data centre these days uses LC, a converter may be required for other connector types on equipment.
SC
Subscriber Connector or Square Connector
Typically found in 1Gbps or 2Gbps Fibre Channel and 1Gbps Ethernet.
Duplex Structured Cabling - Polarity
When running cabling within a DC, there are two methods for configuring cable polarity.
- Single optical paths
- Duplex optical paths
Single paths (e.g. for duplex WAN circuits) may still be presented on a duplex LC interface. These require Port 1 to match Port 1 all the way through the patching structure.
Paired fibres seem to confuse many cabling installers. It could be argued that every patch cable and structured cable is crossed over (i.e. when looking at the LC end, 1 goes to 2 and 2 goes to 1). This has been the cause of failures when deploying new infrastructure on multiple occasions. If a link light doesn't appear when connecting a device to a switch (or device to device), particularly if the structured cabling is new, this is definitely worth checking. If you have access to a visible laser, run it end to end down the optical paths to confirm all paths are configured correctly.
Single Mode vs Multi Mode
A "Mode" refers to the type of light transmission down an optic fibre. Single Mode means there is only one light signal propogated down the centre of the fibre. In Multi Mode communication, light can travel directly down the fibre (without reflection off the sides of the fibre) and light can also travel as reflected signals.
Single Mode | Multi Mode | |
---|---|---|
Diameter of core | Small (8.5 to 10 microns) | Large (50 - 65 microns) |
Cost | High | Low |
Transmission distance | Long (up to many KM) | Short (typically a few hundred metres) |
Typical Connector Colour (not always the case) | Blue | Grey |
Typical Cable Colour | Yellow | Orange or aqua |
Connection reliability | Requires high cost lasers and precision connection, due to small core size | Can use LEDs, can cope with higher connection attenuation |
Wavelengths
Different transmission wavelengths are used, depending on the purpose for the signal. Susceptibility to attenuation and dispersion (which would resuly in signal degradation) is largely dependent on the transmission wavelength.
### IMPORTANT SAFETY NOTE:
Visible light is between 400nm and 700nm. Sometimes it seems that an 850nm SFP transmission is visible (perhaps due to a broader spectrum transmitter than strictly 850nm), however for the wavelengths used in Fibre Channel and Ethernet communications, the primary spectrum (>800nm) is NOT visible to the human eye. Eyesight damage may occur if looking directly at an SFP or an active cable, even though you may not see the light.
An older digital camera (e.g. iphone 4) is susceptible to infrared light and can be used to check for short wavelength light. Otherwise, find another method of testing
Optimal Wavelength windows
Certain windows and bands of wavelengths have been tested and found to be suitable for transmission of light with lower signal loss. The below table was sourced from http://www.rp-photonics.com/optical_fiber_communications.html - these specifications are broader than pure Fibre Channel and Ethernet purposes.
Window | Band | Description | Wavelength Range |
---|---|---|---|
First | Short/Local transmission | 800-900nm | |
Second | O Band | Original | 1260-1360nm |
Second & Third | E Band | Extended | 1360-1460nm |
Third | S Band | Shorter Wavelengths | 1460-1560nm |
Third | C Band | Conventional ("erbium window") | 1530-1565nm |
Third | L Band | Longer Wavelengths | 1565-1625nm |
Third | U Band | Ultra Long Wavelengths | 1625-1675nm |
Some typical wavelengths for Fibre Channel SFPs (based on sfpshow output in fabric OS) are below (u = microns)
SM/MM | SW/LW | Wavelength | Max Speed | Distance (9u) | Distance (50u) | Distance (62.5u) |
---|---|---|---|---|---|---|
Multi Mode | Short Wave | 850nm | 8Gbps | N/A | 50m | 20m |
Single Mode | Long Wave | 1310nm | 8Gbps | 10km | N/A | N/A |
Single Mode | Long Wave | 1510nm | 4Gbps | 40km | N/A | N/A |
Fibre Channel vs Ethernet
The main differenceis speed. Notably, Ethernet seems to progress in base 10 increments (1Gbps, 10Gbps, etc) whereas Fibre Channel progresses in 2Gbps increments (2Gbps, 4Gbps, 8Gbps, etc).
Fibre Channel communications are far more efficient than Ethernet frames as a dedicated fibre channel switched fabric does not have the same challenges to overcome as a broader ethernet network. For starters, Routing isn't required, the protocol stack is thinner (e.g. TCP/UDP ports don't apply), and IP/MAC translation isn't required. Therefore, comparing 1Gbps Ethernet with 1Gbps Fibre Channel isn't really "apples for apples". Still, below are the main speed milestones for each approach:
Year | Fibre Channel | Ethernet |
---|---|---|
1983 | - | 10Mbps |
1995 | - | 100Mbps |
1997 | 1Gbps | - |
1998 | - | 1Gbps |
2001 | 2Gbps | - |
2002 | - | 10Gbps |
2005 | 4Gbps | - |
2008 | 8Gbps | - |
2010 | - | 40Gbps |
2010 | - | 100Gbps |
2011 | 16Gbps | - |
2015 | 32Gbps | - |
2015 | 128Gbps Multi Channel | - |
2016 (est*) | - | 25Gbps |
2017 (est*) | 64Gbps | 400Gbps |
2020 | 128Gbps | - |
Benefits of Fibre Channel
- Dedicated physical network fabric ensures there's no risk of Ethernet traffic interferring with Storage traffic (loss of storage connectivity is often the most catastrophic connection loss event which can occur in IT as it carries a high risk of raw data corruption)
- Better efficiency compared to Ethernet
- Most tape libraries only offer Fibre Channel connectivity
- Highly secure when using LUN masking and FC zoning
Problems with Fibre Channel
- Very high cost
- Each connected device requires dedicated FC HBAs in addition to any Ethernet connectivity (which can quickly consume PCI slots)
- Only block storage is supported (i.e. "FC")
Benefits of Ethernet
- Low cost
- Converged infrastructure - no need to purchase separate switching for Storage or separate HBAs in servers
- Multiple storage presentation protocols supported (iSCSI, CIFS, NFS)
- Routable storage
- Storage can be secured by firewalls & session authentication
Problems with Ethernet
- High risk of interruptions due to competing Ethernet traffic (can be addressed through QOS, VLANs, etc)
- Less efficiency compared to FC
DWDM
"Dense Wave Division Multiplexing"
This is an optical multiplexing technology which facilitates an increase in bandwidth, supporting multiple types of signals across a single optical fibre simultaneously. This is achieved by encoding each individual signal to transmit on a different wavelength.
In the above diagram, Customer A could use the same single long distance optical fibre to connect multiple FC switches between two sites, plus multiple Ethernet switches. Each incoming/local signal (e.g. 850mn) is converted to a DWDM signal and retransmitted on one of the DWDM Channels. The channels are multiplexed and transmitted over the same long distance cable. This technology is widely used by telcos to support multiple customers over the same physical cable, so as to avoid needing to run large amounts of high-cost optical cables, individually for each circuit
GBICs (GigaBit Interface Converters)
These are pluggable modules which convert a signal from copper transmission (for shorter distances) to optical (supporting much longer distances). GBICs support gigabit transmission, therefore are more prevalent in older ethernet switching topologies.
Specifications are normally printed on the GBIC label
In the above image, this GBIC supports transmission up to 80km using a single mode transmitter at 1550nm.
GBICs are available in single-mode and multi-mode, depending on the requirements for the optical circuit.
The optical interface is SC
SFPs (Small Form-factor Pluggable transceivers)
These are smaller than GBICs and support higher transmission speeds. Typically
- Fibre Channel - 4Gbps, 8Gbps, 16Gbps
- Ethernet - 10Gbps
Multiple modes of operation (single-mode, multi-mode, etc) are supported, depending on the SFP (but a single SFP will only support one mode of operation)
Specifications are normally printed on the SFP label
The optical interface is LC
Brocade Fabric OS - Useful Commands to view current configuration
Command | Description |
---|---|
supportshow | Use this to get a detailed set of config data from a switch. Note - it is a very long output, best to redirect it to a text file first. I typically use this for monthly audit/config captures, so I can compare a config to previous months / years (particularly useful when troubleshooting faults) |
sfpshow {port} | Gather detailed specifications and operating parameters for each sfp |
switchshow | Print a summary of the switch and all ports connected (including which WWPNs are connected to each port) |
portshow | Print detailed information on each port. If using NPIV, this will show the individual WWPNs logged in using this port |
chassisshow | Hardware config - for IBM switches this will show both the IBM and Brocade serial numbers |
fabricshow | Print details on each switch connected in the fabric (including the IP addresses of other switches). Similar functionality to cdp neighbour show in Cisco |
islshow | Print detailed information on inter switch links |
cfgshow | Print the zoning and alias configuration |