Network Abstraction Layer (NAL) Architecture

The Network Abstraction Layer (NAL) is a network layer that abstracts the network hardware and driver functions from the upper-level protocol stacks. It provides management interfaces for those upper layers to easily query for and use available network assets and take advantage of network functionality such as EtherCAT, TSN (Time Sensitive Networks), and PTP (Precision Time Protocol). It is a separate protocol layer from the RT-TCP/IP Stack.

The image below shows a high-level layout of the NAL architecture.

Key Components

Network Abstraction Layer (NAL)

The RTX64 Network Abstraction Layer serves a dual purpose. First, it abstracts away details of the NIC devices from the upper-level applications. It does this by providing a RtNal interface for Real-time applications to query what NIC devices, and queues are available for use along with what device configuration and capabilities are available.

Based on this information, an application can choose the network resources that are best for meeting its resource requirements. Once this determination is made, the application can then use the RtNal interface to reserve and attach to the appropriate device interface(s).

The second purpose of the NAL is to host the actual network interface drivers (NIC drivers). This allows for a much more stable system since the hardware drivers are completely decoupled from user applications.

As an abstraction layer, the NAL is a real-time process that loads all configured drivers at launch and provides the information to real-time applications via the RtNal interface.

The RT-TCP/IP Stack runs as a NAL client application, using one receive and one transmit queue of an interface. The Stack uses the default receive queue and the transmit queue with the same number. The remaining interface queues are available to other NAL client applications.

NOTE: If one or both queues needed by the Stack to start an interface are used by another NAL application, the Stack will fail to start the interface and return error code ERROR_ACCESS_DENIED.

Network Interface Drivers

Network interface drivers are RTDLLS that are loaded and initialized by the NAL. Driver functions (RTN and RTND API) execute in the context of the NAL and Real-time applications attached to the NAL.

Understanding the Calling Sequence

RtNal NIC drivers loaded by the RTX64NAL process have a defined interface between the NAL and the driver. The sequences of initialization, transmit, and receive are shown below. The following examples use the RtNalIGB.rtdll driver.

NOTE: In the following diagram, the color red indicates the NAL context. The color gray indicates the context of a NAL application.

Initialization

The RTX64NAL process loads the RTDLL (driver) that corresponds to the NIC. In the below diagram, the RtNal loads and initializes RtNalIGB.rtdll.

  1. The NAL must make the correct calls to the NIC driver, such that the NIC hardware is properly initialized, beginning with a call to RtndInitializeInterface.
  2. When the driver receives this call, it must find the NIC on the PCI/PCIe bus. To do this, the driver calls the Real-Time Network library function RtnEnumPciCards, in the context of RtndInitializeInterface/RTX64NAL process. RtndInitializeInterface returns configuration information for the driver to the NAL. The NAL application can query actual interface configuration by calling RtNalEnumInterfaceInfo.
  3. Next, the RTX64 NAL process calls RtndConfigure and the hardware is initialized and configured.

The below diagram focuses on a network application. It shows the simplest use of a network application in RTX64. At a minimum, the network application queries the NAL for its queues and selects one for use. It is then up to the network application to release those queues once it is done with them. Note that the NAL releases unreleased application queues when the network application terminates. The image below depicts this interaction on the network application with the NAL and, ultimately, the specific network adapter hardware associated with its queues.

Transmitting a Packet

A network application sends Ethernet packets at the data link layer. The data link layer, or layer 2, is the second layer of the seven-layer OSI model of computer networking.

A NAL application can allocate memory for a packet in its own application format that can store some housekeeping data in addition to the packet data buffer. A NAL application can pass the pointer to this packet memory to a RtNalTransmit call, which calls the driver’s RtndTransmit API. The driver then calls RtnDecodePacket which in turn invokes the application decode packet callback fpRtnDecodePacket to get the pointer to the data buffer for the Ethernet packet to transmit, as well as its size. fpRtnDecodePacket is configured via the RtNalConfigureQueue API. The driver then copies the data to a buffer previously allocated by the driver for the DMA transaction. It then communicates the new data to the NIC and returns.

Transmitting Multiple Packets

RtNalTransmitEx transmits multiple Ethernet frames in a single call. An application must pre-allocate NAL frames by calling RtNalAllocateFrame and pass the array of pointers to NAL frames to RtNalTransmitEx. RtNalAllocateFrame returns the pointer to a RTNAL_FRAME structure, which contains virtual and physical addresses of the Ethernet frame. The application must not change these addresses. An application can also provide callbacks to be called with some or each of the frames from a transmit complete thread. This callback pointer is written to the RTNAL_FRAME structure.

NOTE: To use transmit complete callbacks, the transmit complete notifications must be enabled for the queue. RtNalTransmitEx calls RtndTransmitEx in the driver to submit frames to the driver.

The NAL takes ownership of all the frames submitted to the driver. It relinquishes ownership of each frame after the transmission completes in a transmit complete thread. An application should call the RtNalIsApplicationFrame function to query a frame’s ownership.

NOTE: The application must not modify the RTNAL_FRAME structure or the frame data buffer when the NAL frame is owned by NAL.

When a queue is acquired with transmit notifications, the NAL runs an additional transmit complete thread per queue. This thread runs on the transmit complete ideal processor and transmit complete priority configured for the interface. This thread calls the RtndServiceTransmitQueue API exported by the driver.

The function RtndServiceTransmitQueue calls the RtnTransmitComplete callback to transfer the ownership of packets, transmitted by RtNalTransmitEx, back to the application. It then calls an application transmit complete callback.

If an application has not configured transmit complete notifications for the queue, transmitted packets are serviced in the driver’s transmit complete thread. This thread also transfers the ownership of packets, transmitted by RtNalTransmitEx, back to the application but it does not call an application callback.

NOTE: In the following diagram, the color red indicates the NAL context. The color gray indicates the context of a NAL application.

Receiving Packets

A network application can also receive Ethernet packets at the data link layer (Layer 2). To do this, it creates a thread that waits for a signal indicating receipt of multiple packets. When the NIC generates an interrupt to indicate the receipt of multiple packets, the driver’s Interrupt Service Thread (IST) notifies the network application by calling RtnNotifyRecvQueue, which in turn sets the signal that the thread is waiting on. The application receive thread should call RtNalGetReceivePacketCount to know how many packets are waiting and then call RtNalReceivePacket in a loop. RtNalReceivePacket calls the driver’s RtndReceive function to retrieve the waiting packet. The driver calls RtnGetPacket, which in turn invokes an application registered get packet callback to retrieve a packet from the application. The driver then calls RtnDecodePacket, which in turn invokes the application registered decode packet callback to get the pointer to the application data area. Lastly, the driver copies the data from its DMA buffer to the application data buffer and returns.

Pointers to the application get packet and decode packet callbacks are supplied in the capabilities structure in the RtNalConfigureQueue function.

Instead of using RtNalGetReceivePacketCount, a NAL application can call RtNalReceivePacket in a loop until it fails with error code ERROR_NO_DATA error code. The RT-TCP/IP Stack uses this method.

Receiving Packets in a Callback

The pointer to an application receive packet callback can be passed to the driver in fpRtnReceiveCallback in structure RTNAL_QUEUE_CAPABILITIES passed to RtNalConfigureQueue. A non-zero pointer to the callback function must be provided if an application uses RtNalReceiveWithCallback or the receive polling timer.

The application receive packet callback function is called directly from RtndReceiveWithCallback in a driver. RtNalReceiveWithCallback does not access the application-defined packet structure and does not copy data; the application must process the packet in the receive packet callback.

NOTE: In the following diagram, the color red indicates the NAL context. The color gray indicates the context of a NAL application.

If the application uses a polling timer, it must acquire the receive queue without receive notifications and does not run the receive thread. The polling timer can be started by the function RtNalEnableReceivePolling and stopped by the function RtNalDisableReceivePolling. The polling timer handler calls the driver’s function RtndReceiveWithCallback.

See Also: