COMPLETE DATA RECOVERY SERVICE
|
COMPLETE DATA RECOVERY SERVICE
|
|
|
Fibre Channel TutorialThe Fibre Channel Bench Reference (reference book)
Fibre Channel is a computer communications protocol designed to meet the many requirements related to the ever increasing demand for high performance information transfer. The goals of Fibre Channel include:
In general, Fibre Channel attempts to combine the benefits of both channel and network technologies. A channel is a closed, direct, structured, and predictable mechanism for transmitting data between relatively few entities. Typically, once a channel is set up, there is very little decision making needed, thus allowing for a high speed, hardware intensive environment. Channels are commonly used to connect peripheral devices such as a disk drive, printer, tape drive, etc. to a workstation. Common channel protocols are Small Computer System Interface (SCSI) and High Performance Parallel Interface (HIPPI). Networks, however, are unstructured and unpredictable. Networks are able to automatically adjust to changing environments and can support a larger number of connected nodes. These factors require that much more decision making take place in order to successfully route data from one point to another. Much of this decision making is done in software, making networks inherently slower than channels. Examples of common networks are Ethernet, Token Ring, and Fiber Distributed Data Interface (FDDI). In 1988, the American National Standards Institute (ANSI) Task Group X3T11 began work on the Fibre Channel standards. At this time, at least 20 separate documents, each defining an aspect of Fibre Channel, are either proposed or in draft form. Although several of the standards are in their early stages, Fibre Channel products are being shipped today. Fibre Channel is making the biggest impact in the storage arena, in particular, using SCSI as an upper layer protocol. Compared with traditional SCSI, the benefits of mapping the SCSI command set onto Fibre Channel include:
Essentially, Fibre Channel, using the Arbitrated Loop topology, is simply being used as a replacement for SCSI. Many companies are already shipping SCSI adapter cards for several platforms and operating systems, as well as disk drives and storage devices with Fibre Channel interfaces. There is currently interest in connecting SCSI Tape devices as well. In addition to using SCSI, several companies are selling Fibre Channel devices that run Internet Protocol (IP). Although the ULP is independent of the topology, IP is more commonly found in switched Fabric environments. At the present time, SCSI and IP are pretty much the only two ULPs commercially used on Fibre Channel. Fibre Channel will most likely continue to expand into the storage markets, which will make use of its benefits over traditional channel technologies such as SCSI. Being able to access mass storage devices quicker and from greater distances is very attractive to such applications as multimedia, medical imaging, and scientific visualization. Because of the greater distances allowed, Fibre Channel has advantages in disaster recover situations as storage devices can be placed remotely. Unfortunately, the current trend in Fibre Channel is to continue to define more and more standard documents that increase the complexity of the protocol. This is probably the biggest threat to its future. It is often easier to understand a communications protocol if it is first broken down into parts or layers. Some people are familiar with the International Standards Organization (ISO) - Open Systems Interconnect (OSI) model, which breaks the communications puzzle into seven layers: Physical, Data Link, Network, Transport, Session, Presentation, and Application. Fibre Channel does not follow this model, but instead, the protocol has been broken into five layers: FC-0, FC-1, FC-2, FC-3, and FC-4. Each is briefly described below along with the main functions it defines. FC-0
FC-1
FC-2
FC-3
FC-4
FC-0 and FC-1 can be thought of as defining the Physical Layer of the OSI model. FC-2 is similar to what other protocols define as a Media Access Control (MAC) layer, which is typically the lower half of the Data Link layer. Fibre Channel, however, does not define the concept of a MAC. FC-3 is not really a layer at all. It is still a largely undefined set of services for devices having more than one port. An example is striping, where data is transmitted out of all ports at the same time in order to increase bandwidth. FC-4 defines how other well-known higher layer protocols are mapped onto and transmitted over Fibre Channel. Thus, one can roughly think of the Fibre Channel layers defining up through the Transport layer of the OSI model. Fibre Channel without a doubt wins the award for the largest and most complex set of standards documents for a communications protocol. The initial effort was started back in 1988, and much of it continues to undergo significant development to this day. There are now well over 20 individual standards documents, some of which have been adopted as standards, but many remain in draft form. The standards body under which the documents are being developed is the American National Standards Institute (ANSI). The main document by which all others are based is FC-PH (ANSI X3.230-1994, Fibre Channel - Physical and Signaling Interface). It defines the FC-0, FC-1, and FC-2 layers. In addition to FC-PH, two other Physical and Signaling Interface documents have been developed: FC-PH-2 (ANSI X3.297-1997, Fibre Channel - Physical and Signaling Interface - 2) and FC-PH-3 (Project 1119-D, Fibre Channel - Physical and Signaling Interface - 3). These add various functions and features to the original FC-PH. Another key document is FC-AL (ANSI X3.272-1996, Fibre Channel - Arbitrated Loop). This describes the behavior of an Arbitrated Loop device. The Arbitrated Loop topology was an afterthought to the original design of Fibre Channel. As a result, it was not defined in FC-PH, but instead became its own standard. It extends the rules and definitions of FC-PH to accommodate the topology. FC-AL-2 (Project 1133-D, Fibre Channel - Arbitrated Loop - 2) and FC-AL-3 (unassigned, Fibre Channel - Arbitrated Loop - 3) are being developed to expand on the original. Several documents exist that describe how ULPs are to be mapped onto Fibre Channel:
There are many more projects in various stages of completion such as:
Fibre Channel defines three topologies, namely Point-to-Point, Arbitrated Loop, and Fabric. Each of these is described below. A Point-to-Point topology is the simplest of the three. It consists of two and only two Fibre Channel devices connected directly together. The transmit fibre of one device goes to the receive fibre of the other device, and vice versa. There is no sharing of the media, which allows the devices to enjoy the total bandwidth of the link. A simple link initialization is required of the two devices before communication can begin.
Arbitrated Loop has become the most dominant Fibre Channel topology, but it is also the most complex. It's a cost-effective way of connecting up to 127 ports in a single network without the need of a Fabric switch. Unlike the other two topologies, the media is shared among the devices, limiting each device's access. Not all devices are required to operate on an Arbitrated Loop; the added functionality is optional. Thus, for a Loop to operate, all devices must be Loop devices.
Arbitrated Loop is not a token-passing scheme. When a device is ready to transmit data, it first must arbitrate and gain control of the Loop. It does this by transmitting the Arbitrate (ARBx) Primitive Signal, where x = the Arbitrated Loop Physical Address (AL_PA) of the device. Once a device receives its own ARBx Primitive Signal, it has gained control of the Loop and can now communicate with other devices by transmitting an Open (OPN) Primitive Signal to a destination device. Once this happens, there essentially exists point-to-point communication between the two devices. All other devices in between simply repeat the data. If more than one device on the Loop is arbitrating at the same time, the x values of the ARB Primitive Signals are compared. When an arbitrating device receives another device's ARBx, the ARBx with the numerically lower AL_PA is forwarded, while the ARBx with the numerically higher AL_PA is blocked. Thus, the device with the lower AL_PA will gain control of the Loop first. Once that device relinquishes control of the Loop, the other device can have a chance. Unlike token-passing schemes, there is no limit on how long a device may retain control of the Loop. This demonstrates the "channel" aspect of Fibre Channel. There is, however, an Access Fairness Algorithm, which prohibits a device from arbitrating again until all other devices have had a chance to arbitrate. The catch is that the Access Fairness Algorithm is optional. Initialization Before the Loop is usable, it must be initialized so that each port obtains an Arbitrated Loop Physical Address (AL_PA), a dynamically assigned value by which the ports communicate. It maps to the lowest byte of the Native Address Identifiers (D_ID and SI_ID). Although the AL_PA is a byte, only 127 values are valid (neutral running disparity). If more than 127 devices are present on the Loop, some will not be able to select an AL_PA and will be forced into non-participating mode. Without getting into all the details of Loop initialization, the basics are as follows. The LIP Primitive Sequence begins the process. LIP is transmitted by an L_Port after it powers on, or when it detects Loop Failure (loss of synchronization at its receiver). The LIP will propagate around the Loop, triggering all other L_Ports to transmit LIP as well. At this point, the Loop is not usable. The next major step is to select a Loop master that will control the process of AL_PA selection. This is done by the L_Ports constantly transmitting Loop Initialization Select Master (LISM) frames. The process is designed so that if an Fabric is present, it will become Loop master (by receiving back its own LISM frames), otherwise, the port with the numerically lowest Port Name will win. All other L_Ports propagate the higher priority LISM frames. The third step is to allow the L_Ports to select an AL_PA. The concept of an AL_PA bitmap is used, where each L_Port selects (and sets) a single bit in the bitmap of a frame originated by the Loop master and repeats the frame back on the Loop. There are 127 available bits, corresponding to the 127 valid AL_PAs. This process is done using four frames, breaking the selection down according to priority. The following table explains:
So for example, if an L_Port had a valid AL_PA before the Loop began this initialization, it will attempt to reclaim this previously acquired value by looking for that bit to be available in the LIPA frame. If it is, it will set the bit and repeat the frame. If it is not available (already been claimed), the L_Port will wait for the LISA frame to come around and claim one there. Once the LISA frame has come back to the Loop master, all L_Ports (hopefully) have selected an AL_PA. Two additional frames may be sent by the Loop master, but only if all L_Ports on the Loop support them. The first is the Loop Initialization Report Position (LIRP). As the frame traverses the Loop, each port adds its AL_PA to the end of a list. When done, the relative positions of all L_Ports is known. Finally, the Loop Initialization Loop Position (LILP) frame is transmitted, which simply allows all L_Ports to look at the finished list. Whether or not LIRP and LILP are used, the Loop master transmits the CLS Primitive Signal to let each port know that the process has finished. At this point, the Loop has finished initializing and ready to be used. Hubs Like most ring topologies, life is made easier if the devices can be connected to a central hub or concentrator. The cabling is easier to deal with, and the hub can usually determine when to insert or de-insert a device. Thus, a "bad" device or broken fibre won't keep the whole network down. Fibre Channel has hubs as well. Most hubs will simply look for valid signal on a port before it will insert the port in the data path. Some hubs are "smarter", and will provide the user with information on each of its ports, allow the user to transmit from the ports, and be more selective as to when ports are inserted. The Fabric topology is used to connect many (224) devices in a cross-point switched configuration. The benefit of this topology is that many devices can communicate at the same time; the media is not shared. Of course, it also requires the purchase of a switch.
When the N_Ports log into the Fabric, the Fabric will assign Native Address Identifiers (S_ID). Other functions of the Fabric include a multicast server, broadcast server, alias server, quality of service facilitator, and directory server. Some Fabrics have FL_Ports, allowing Arbitrated Loops to be connected to the Fabric. Fibre Channel defines several types of ports. They can be grouped a couple of ways. First, any port on a node device, e.g., a disk, a PC, is an N_Port as compared with a port on a Fabric, which is an F_Port. At the same time, any port that happens to have Arbitrated Loop capabilities, it is called an L_Port. Combining these groupings, we also have NL_Ports and FL_Ports. Ports that connect switches together are called E_Ports. They generally do not need to follow Fibre Channel rules and protocols. A port that can act as either an E_Port or an F_Port is called a G_Port. Finally, a G_Port with Loop capabilities, is a GL_Port. Fibre Channel offers a very wide range of media speeds. One of the goals of Fibre Channel is to allow HIPPI to map to it. HIPPI is a 100MB/s technology, thus Fibre Channel's primary data rate allows for data to travel 100MB/s. After factoring in 8B/10B encoding, frame headers, and other overhead, the transmission speed is 1063 Mb/s. This speed is referred to as full speed. There also exists half speed, quarter speed, and eighth speed. In addition, double and quadruple speeds are defined. The following table illustrates. By far, the most common speed is full speed, with some quarter speed devices also in existence.
Despite the name, Fibre Channel can run over both copper and fiber media. The main tradeoffs are that although longer distances can be achieved with fiber, it is more expensive. Speeds up to 100MB/s can run on both copper and fiber; 200MB/s and 400MB/s require fiber media. For copper, the following cable types are used: video cable, miniature cable, and shielded twisted pair. The most common by far is shielded twisted pair, using a DB-9 connector. For fiber, the choices are: 62.5mm multi-mode, 50mm multi-mode, and single-mode. The SC connector is used. Also for fiber, both long wave and short wave lasers can be used. Short wave seem to be most popular now. The short wave transmitters have the option to implement the open fibre control (OFC). This is a scheme designed to prevent disconnected fibers from continuously transmitting light, as a safety precaution. There is a simple protocol where each transmitter on a link periodically transmits short pulses of light to the other receiver. When the receivers detect this, the transmitters can operate normally. Most devices however, are non-OFC. The concept of flow control deals with the problem where a device receives frames faster than it can process them. When this happens, the result is that the device is forced to drop some of the frames. Fibre Channel has a built-in flow control solution to this problem. The idea is simple enough. A device can transmit frames to another device only when the other device is ready to accept them. Before the devices can send data to each other, they must login to each other. One of the things accomplished in login is establishing credit. Credit refers to the number of frames a device can receive at a time. This value is exchanged with another device during login, so each knows how many frames the other can receive. After enough frames have been transmitted and credit runs out, no more frames can be transmitted until the destination device indicates it has processed one or more frames and is ready to receive new ones. Thus, no device should ever be overrun with frames. Fibre Channel uses two types of flow control, buffer-to-buffer and end-to-end. This type of flow control deals only with the link between an N_Port and an F_Port or between two N_Ports. Both ports on the link exchange values of how many frames it is willing to receive at a time from the other port. This value becomes the other port's BB_Credit value and remains constant as long as the ports are logged in. For example, when ports A and B log into each other, A may report that it is willing to handle 4 frames from B; B might report that it will accept 8 frames from A. Thus, B's BB_Credit is set to 4, and A's is set to 8. Each port also keeps track of BB_Credit_CNT, which is initialized to 0. For each frame transmitted, BB_Credit_CNT is incremented by 1. The value is decremented by 1 for each R_RDY Primitive Signal received from the other port. Transmission of an R_RDY indicates the port has processed a frame, freed a receive buffer, and is ready for one more. If BB_Credit_CNT reaches BB_Credit, the port cannot transmit another frame until it receives an R_RDY. End-to-End flow control is not concerned with individual links, but rather the source and destination N_Ports. The concept is very similar to buffer-to-buffer flow control. When the two N_Ports log into each other, they report how many receive buffers are available for the other port. This value becomes EE_Credit. EE_Credit_CNT is set to 0 after login and increments by 1 for each frame transmitted to the other port. It is decremented upon reception of an ACK Link Control frame from that port. ACK frames can indicate the port has received and processed 1 frame, N frames, or an entire Sequence of frames. Fibre Channel defines several communication strategies called Classes of service. The Class used greatly depends on the type of data to be transmitted. The major difference between among the Classes is the types of flow control used. If two N_Ports are to communicate or if an N_Port is to successfully log into a Fabric, there must be at least 1 common Class of service supported between them, since Sequences and Exchanges must take place using a single Class of service. This information is exchanged during Fabric Login and N_Port Login. In Class 1, a dedicated connection is established between two N_Ports. Once established, the two N_Ports may communicate using the full bandwidth of the connection; no other network traffic affects this communication. Due to this, frames are guaranteed to arrive in the order in which they were transmitted. In addition, the media speeds must be the same for all links which make up the dedicated connection. Because of the nature of the dedicated connection, there is no need for buffer-to-buffer flow control; the Fabric does not need to buffer the frames as they are routed. Thus, only end-to-end flow control is used in Class 1. Class 1 would be used when the data needs to be continuous and time critical, such as voice or video. Intermix Intermix is an option of Class 1 whereby Class 2 and Class 3 frames may be transmitted at times when Class 1 frames are not being transmitted. The Class 2 and Class 3 frames may or may not be destined to the same N_Port as the Class 1 frames. Both N_Ports as well as the Fabric must support Intermix for it to be used. Class 2 is referred to as multiplex due to the fact that it is a connectionless Class of service with notification of delivery and non-delivery of frames. Since no dedicated connection needs to be established, a port can transmit frames to and receive frames from more than one N_Port. As a result, the N_Ports share the bandwidth of the links with other network traffic. Frames are not guaranteed to arrive in the order in which they were transmitted, except in the point-to-point or Loop topologies. Also, the media speeds may vary for different links which make up the path. Both buffer-to-buffer and end-to-end flow control are used in Class 2. Class 2 is more like typical LAN traffic, such as IP or FTP, where the order and timeliness of delivery is not so important. Class 3 is very similar to Class 2. The only exception is that it only uses buffer-to-buffer flow control. It is referred to a datagram service. Class 3 would be used when order and timeliness is not so important, and when the ULP itself handles lost frames efficiently. Class 3 is the choice for SCSI. Class 4 provides fractional bandwidth allocation of the resources of a path through a Fabric that connects two N_Ports. Class 4 can be used only with the pure Fabric topology. One N_Port will set up a Virtual Circuit (VC) by sending a request to the Fabric indicating the remote N_Port as well as quality of service parameters. The resulting Class 4 circuit will consist of two unidirectional VCs between the two N_Ports. The VCs need not be the same speed. Like a Class 1 dedicated connection, Class 4 circuits will guarantee that frames arrive in the order they were transmitted and will provide acknowledgement of delivered frames (Class 4 end-to-end credit). The main difference is that an N_Port may have more than one Class 4 circuit, possibly with more than one other N_Port at the same time. In a Class 1 connection, all resources are dedicated to the two N_Ports. In Class 4, the resources are divided up into potentially many circuits. The Fabric regulates traffic and manages buffer-to-buffer flow control for each VC separately using the FC_RDY Primitive Signal. Intermixing of Class 2 and 3 frames is mandatory for devices supporting Class 4. The idea for Class 5 involved isochronous, just-in-time service. However, it is still undefined, and possibly scrapped altogether. It is not mentioned in any of the FC-PH documents. Class 6 provides support for multicast service through a Fabric. Basically, a device wishing to transmit frames to more than one N_Port at a time sets up a Class 1 dedicated connection with the multicast server within the Fabric at the well-known address of hex'FFFFF5'. The multicast server sets up individual dedicated connections between the original N_Port and all the destination N_Ports. The multicast server is responsible for replicating and forwarding the frame to all other N_Ports in the multicast group. N_Ports become members of a multicast group by registering with the Alias Server at the well-know address of hex'FFFFF8'. The Class 6 is very similar to Class 1; Class 6 SOF delimiters are the same as used in Class 1. Also, end-to end flow control is used between the N_Ports and the multicast server. Unlike many LAN technologies that use a fixed six-byte Media Access Control (MAC) address, Fibre Channel uses a three byte address identifier, which is dynamically assigned during Login. N_Ports transmit frames from their own Source_ID (S_ID) to a Destination_ID (D_ID). Addresses in the range of hex'FFFFF0' to hex'FFFFFE' are special, well-known addresses uses for such things as the Fabric, Alias Server, or the Multicast Server. Before Fabric Login, the N_Port's S_ID is undefined: hex'000000'. Hex'FFFFFF' is reserved for broadcast. In a point-to-point topology, Fabric Login will fail of course, and the two ports will simply chose two unique addresses. Arbitrated Loop devices still use the three byte address identifiers, but also use an Arbitrated Loop Physical Address (AL_PA). AL_PAs are one byte values dynamically assigned each time the Loop is initialized. Once the Loop is initialized and (hopefully) each L_Port has selected an AL_PA, public NL_Ports will attempt Fabric Login. If there is an FL_Port, the Fabric will assign the upper two bytes of the NL_Port's address identifier and usually allow the low byte to be the NL_Port's AL_PA. (If not, the Loop will need to be re-initialized so the NL_Port can select the Fabric assigned AL_PA). If no Fabric exists or if an NL_Port is a private NL_Port (does not login with the Fabric), the upper two bytes of the address identifier will remain '0000', and the lower byte will simply be the NL_Port's AL_PA. But there still needs to be a way of uniquely identifying a port - even for much of the above initialization to take place. This is accomplished using Name_Identifiers, a fixed 64-bit value. Name_Identifiers are used to uniquely identify nodes (Node_Name), a Port (Port_Name), and a Fabric (Fabric_Name). Name Identifiers are not used to route frames, but are used in mapping to upper layer protocols. Fibre Channel defines two types of login procedures, Fabric and N_Port. With the exception of private NL_Ports, all other node ports must attempt to log in with the Fabric. This is typically done right after the link or the Loop has been initialized. It consists of the node port transmitting an Fabric Login (FLOGI) frame to the well-known Fabric address hex'FFFFFE'. The normal response is an Accept (ACC) frame from the Fabric back to the node port. Fabric Login accomplishes the following things:
Before a node port can communicate with another node port, it must first perform N_Port Login with that node port. Similar to Fabric Login, the process entails transmitting a PLOGI frame to the destination node port. Again, the normal response is an ACC frame. N_Port Login accomplishes the following things:
Both Fabric Login and N_Port Login are intended to be long-lived. Once logged in, a device can stay logged in indefinitely, even if it has no further data to transmit at that time. The easiest way to understand the methods by which information is transmitted over Fibre Channel is by looking at the problem in the form of a hierarchy. At the lowest level, Fibre Channel uses the IBM 8B/10B encoding scheme. Basically, every byte of data that is to be transmitted is first converted into a 10-bit value called a Transmission Character. Using this encoding scheme has the following benefits:
Fibre Channel defines a "1" to simply be the state with more optical power (for optical links), or the state where the "+" pin is more positive than the "-" pin (in the case of copper). The 8B/10B encoding uses the idea of running disparity, which is concerned with the number of 1s and 0s in each Transmission Character. Running disparity is evaluated after the first 6 bits of each Transmission Character and again after the last 4 bits and can be either positive (more 1s than 0s) or negative (more 0s than 1s). It is desirable to try and equalize the number of 1s and 0s over time. Thus, every byte to be transmitted is encoded into one of two 10-bit representations depending on the current running disparity. As stated, every byte to be transmitted is first converted into a 10-bit Transmission Character. But there are many more possible 10-bit Transmission Characters than are needed to map to particular bytes. Most of the remaining 10-bit encodings are not defined, and only one is used. This is the special K28.5 Transmission Character, which contains the "comma", a 7-bit string that cannot occur in any Data Transmission Character. Because of this, the K28.5 is used as a special control character. All information in Fibre Channel is transmitted in groups of four Transmission Characters called Transmission Words. Some Transmission Words have K28.5 as the first Transmission Character. These are special Transmission Characters called Ordered Sets. Some Ordered Sets mark the beginning and end of frames (frame delimiters). Others convey information in between frames in the form of Primitive Signals (a single Ordered Set) and Primitive Sequences (a stream of the same Ordered Set). Examples of Ordered Sets are: Start of Frame (SOF), End of Frame (EOF), Idle, Receiver_Ready (R_RDY), Loop Initialization Primitive (LIP), Arbitrate (ARB), Open (OPN), Close (CLS), and several others. Fibre Channel defines a variable length frame consisting of 36 bytes of overhead and up to 2112 bytes of payload for a total maximum size of 2148 bytes. The total size of the frame must be an even multiple of four bytes so that partial Transmission Words are not sent. Between 0 and 3 pad bytes are appended to the end of the payload to satisfy this rule. A Start of Frame (SOF) delimiter and End of Frame (EOF) delimiter mark the beginning and end of each Fibre Channel frame. The Cyclic Redundancy Check (CRC) is the same 32-bit CRC used in FDDI. A Fibre Channel Sequence is a series of one or more related frames transmitted unidirectionally from one port to another. All frames must be part of a Sequence. Frames within the same Sequence have the same SEQ_ID field in the header. The SEQ_CNT field identifies individual frames within a Sequence. For each frame transmitted in a Sequence, SEQ_CNT is incremented by 1. This provides a means for the recipient to arrange the frames in the order in which they were transmitted and to verify that all expected frames have been received. Multiple Sequences to multiple ports may be active at a time. A Fibre Channel Exchange is a series or one or more nonconcurrent Sequences between two ports. The Sequences may be in either direction. All Sequences (and therefore all frames) must be part of an Exchange. The originator of the Exchange assigns the OX_ID field. The responder assigns the RX_ID field. As another perspective, one can use the following analogy: frame ~ word Of course, one main difference is that a Fibre Channel device can "speak" more than one sentence and hold more than one "conversation" at a time.
Source : http://www.iol.unh.edu/training/fc/fc_tutorial.html
|
You can upgrade your
browser from two of the three links below.
|
