SMB

SMB2

SMB3

CIFS

SMB most important features

January 2, 2019 by Tal Widerman

SMB, or Server Message Block, is one of the pillars of mass data transfers across networks. In the age of data centers and virtualized servers, this is the protocol that is doing the hard lifting, by moving, copying and modifying terabytes of user data, and keeping it secure and encrypted from hackers and ransom attacks.

The protocol itself has undergone a rapid evolution from its early days, and the latest Microsoft SMB 3.1.1 version is aimed at speed, flexibility and extreme security.

For virtualized data centers, the SMBv3.x family of dialects is the de-facto standard for high performance, offering a rich set of functions that weren’t available earlier.

How SMB works

Everyone knows how easy it is to map network drives on one’s own device, and then access files and folders as if they were kept locally. In the background, what makes this possible is the SMB protocol. SMB is a protocol, just like the http protocol we use to browse the web, a set of rules that defines how data access is to take place.

When two devices wish to share data with each other on a network, they do so with the help of network redirectors. All such connection and access requests are handled and completed by redirectors, and these are sent as “data packets”. The data packets are of three types: session control packets, file access packets and general message packets. SMB works at Layer 7, the application layer, and uses TCP/IP on port 445.

The first SMB protocol, SMB1 used 16-bit data sizes as well as small data buffers, and thus had limited power. With SMB2 onwards, the packet size has increased to 32-bit and even 128-bit for file handles. The new SMB versions are thus faster when it comes to transferring large files over networks, and also allow caching, durable connections and many other features.

Key SMB features

  1. Authentication
    1. NTLM
    2. NTLMv2
    3. Kerberos
    4. Pre-Authentication Integrity
  2. Secured Data Transfer
    1. Message signing
    2. SMB Encryption
  3. Performance
    1. Concurrent operations
    2. Maximum Transmission Unit
    3. Multichannel Technology
    4. SMB Direct and RDMA
  4. Durability and Reliability
    1. SMB Lease and Oplock
    2. Durable handles and Persistent handles
    3. Scalability
    4. Continuous Availability Service
    5. SMB Witness - SWN
    6. Transparent Failover
    7. Scale Out File Servers

 

1. Authentication

NTLM

Authenticating the client or user identity is important before file shares are given access. The NT Lan Manager is one of the two security protocols that offers authentication to hosts in a Windows network. For example, before an SMB connection is made, clients get authenticated by their domain where the server is located.

NTLM uses a challenge/response method to authenticate users and devices. The method incorporates a three-way handshake and password hashing to prove user authenticity; however the cryptography for this is weak, and vulnerable to ‘pass-the-hash’ attacks.

NTLMv2

The second version of NTML is more secure with strong cryptography, and can prevent spoofing attacks. This protocol is still widely used as it is compatible with old systems.

The complete NTLM protocol suite is a single implementation comprising both NTML versions along with NTML2 Session protocol.

Kerberos

Kerberos, developed by MIT, is a network authentication protocol that works transparently with the Windows Active Directory, and was introduced in Windows 2000.

The Active Directory of Windows is used to manage users, computers and services that are part of the network hierarchy. The server that runs the Active Directory Domain Service becomes the domain controller, able to authenticate and authorize network users and enforce security policies.

Kerberos employs cryptography to allow clients and servers to prove their identity to each other. Stronger than NTML, Kerberos authenticate happens separately from SMB using tickets supplied by a Kerberos Key Distribution Center (KDC). Clients and servers use these tickets and authenticate themselves during connections on insecure networks.

Kerberos also allows encrypting subsequent communication with secret-key cryptography after the identities of clients and servers are established.

Pre-Authentication Integrity

With SMB2.0.2 (Vista) onwards, the protocol ensures that users and clients are authenticated when they connect to servers. The security has been further enhanced in SMB3.0 (Windows 8) with a new algorithm for signing data packets.

The main function of pre-authentication integrity is to prevent what are called ‘man-in-the-middle’ attacks, whereby hackers attempt to inspect or eavesdrop on data packets and tamper with the connection and authentication properties. Using cryptographic SHA-512, this feature verifies requests for setting up sessions and subsequent negotiations.

 

2. Secured Data Transfer

Message signing

SMB allows for digital signing of data packets. Thus users who receive data packets can be assured of their point of origin and authenticity. Digital signing has been introduced to prevent ‘man in the middle’ attacks and tampering.

SMB Signing can be activated on all supported Windows versions, and is a default feature for domain controllers. On domain controllers, all users can thus download authentic group policies.

A new hashing algorithm, HMAC SHA-256, makes SMB2.0 more secure compared to the earlier dialects. With SMB3.0, security has been further enhanced by the AES-CMAC algorithm, and with Windows 10, AES-128-GCM has been introduced. The new algorithms also improve performance, especially on WANs.

SMB Encryption

Abandoning the older SMB versions for the more secure SMB3.0 is a great advance in network security. Although there are solutions such as IPSec, high security can be achieved and costs reduced by simply implementing the SMB 3.x protocol, where one needs to only check a box.

Encryption has been introduced to protect data in transit from malicious ransomware and other hacking attempts. The SMB3.X protocols allow setting up encryption on either single shares or file servers as per need.

From SMB3 onwards, it is now possible to detect ‘man-in-the-middle’ attacks and simply disconnect the network. The feature can be activated using the File Server Manager or via the powershell.

SMB encryption is of great importance for mobile workers, who work from unsecured networks, and is valuable for protecting sensitive corporate data during transfer. The feature requires both the client and the server to use SMB3.x protocols.

SMB3.0 uses the AES-CCM algorithm for encryption. Data integrity validation is done by using AES-CMAC algorithm. The older HMAC-SHA256 used by SMB 2 is no longer in use. The new algorithms work fast on modern CPUs using AES instruction support.

In SMB3.1.1, data packet confidentiality has been further strengthened with the addition of AES-128-GCM in Windows 10 and Windows Server 2016. This also comes with a performance increase of up to two times. Security is now stronger against tampering and eavesdropping with the use of cipher negotiation when a connection is established.

 

3. Performance

Concurrent operations

Users often work together and may require simultaneous access to files stored on servers. SMB allows locking files as well as giving concurrent access.

When a user wants access to a file that could be shared, the Lock feature is brought into action. Lock allows the user to perform some actions on a shared local file without notifying the server. It can also notify the server when only one client is present, or when only the read action is being performed on a shared file.

Concurrent operation lets a user ask for specific access, such as granting read-only or write-only access. The SMB server keeps track of all such requests.

Maximum Transmission Unit

Microsoft has introduced new features to increase performance of large networks such as a 10 gigabit Ethernet. In SMB 2.1, better network speed can be obtained with large, multi-credit operations, also called the Maximum Transmission Unit, or MTU. The MTU is the size of the biggest data unit that can be sent across the protocol on a network.

By increasing the size of MTU, the maximum data unit can be 1MB, which allows for faster file transfer, and reduces the number of packets sent. MTU was introduced with Windows 2008 R2 and Windows 7.

MTU enhance data usage performance when for example, query an SQL server Database, making copies of virtual hard disks (Hyper-V), back-up and restoring data.

Multichannel Technology

Much faster file transfers are possible with the multi-channel feature of SMB3.x. This can be done by combining several NIC cards, and all it takes is plugging them into the network.

Multi-channeling works by combining the bandwidth of several networking cards, and allows CPU cores to split data streams for faster data transfers. For example, a client with several 1 gigabit cards can connect faster to a server with a large, 10 gigabit card, or vice versa. Both the client and server can be equipped with large NIC adapters, and thus utilize their CPU cored to the maximum.

Advantages of SMB multi-channel feature:

  • Increased network performance outside Windows clustering
  • Multiple data paths available
  • Higher throughput and network fault tolerance
  • Automatic configuration (dynamic addition of connections discovered automatically)

Multi-channel features are available only with SMB3.x, while older protocols can only use one SMB connection.

With Windows 10 update of 2016, SMB multi-channeling is much easier, as there is no need to add machine names and IP addresses.

SMB Direct and RDMA

SMB Direct and Remote Direct Memory Access (RDMA) make for a faster and more efficient clustered storage environment. RDMA allows for a quick, memory-to-memory transfer of data. All it takes is linking the servers using networking hardware such as InfiniBand, iWARP or RoCE.

In a typical SOFS system as described later, several Windows file servers are grouped together to share files to workload servers. A failure of one server is managed by quickly restoring the connection using transparent failover. A bottleneck that can constrict speed lies with how the storage devices connect to the servers.

Using Ethernet networks with even 10Gbps isn’t fast enough for enterprise data management requirements. Storage devices (SAN, FC, iSCSI) are commonly tied up into pools, and here virtual disks can be created for use as Hyper-V clusters, or in the case of SQL servers, file shares for accessing the database. These workload hosts receive their connection with SOFS servers via SMB3.x protocols.

With the RDMA feature of SMB3.x, a high speed data network can be set up. These are also called high performance computing (HPC) environments, often found in systems for processing financial or scientific data. With remote direct memory access, CPU load is minimized as well as latencies in networking.

With RDMA, a network file server can thus act as local storage when using Microsoft Hyper-V or SQL Server 2012. This feature is available only from SMB3.0 and above.

 

4. Durability and Reliability

SMB Lease and Oplock

Lease is a new caching feature introduced in Windows 7. Similar to Oplock, lease allows clients to better utilize the network and adjust buffering policy. Although the names of lease types are different, they are similar to Oplock types. Some main lease types are read-caching, write-caching, and handle-caching.

The Oplock feature of SMB gives it many advantages such as file caching, simultaneous access, synchronization of the cache to reduce round trips, and thus overall better file sharing performance. A type of file locking, Oplock is actually a request rather than a command, and sent by users to servers.

The request is met on certain conditions that have been allowed. Once the lock is invalid, the server sends a break to the user. Oplocks can be batch, exclusive or level 2 locks.

Durable handles and Persistent handles

How does SMB handle temporary loss of connection when a file is still open? From SMB2.0 onwards, durable handles are open file handles that survive a short disconnect. Clients can then re-connect after the network is restored. The persistence is achieved with the oplock/lease feature, which reconnects to the file.

To make the connections more resilient and long lasting, persistent handles are used. Thus in a 60-second window, the client can easily re-establish connection while also denying a third party to connect to the file/s.

Persistent handles are able to do this as they get mirrored on a separate storage (SFO partner), from where they are reclaimed. The SMB connections are thus continuously available and there is a transparent reconnect for smooth and uninterrupted operations.

Scalability

SMB allows for a great increase in handling number of users, and giving them file access and operational control for every server used.

The new SMB dialects are built for scaling up, and allow every server to handle greater numbers of open file handles, file shares and concurrent operations.

The Scale-Out File Services is a new feature ideally suited to data centers, whereby file storage can be made continuously available with the addition of new servers.

Continuous Availability Service

An important feature for data centers and services, continuous availability is an enterprise grade feature of SMB3.0. The new SMB servers support clustered servers or scale-out servers. File shares in these networks can be set to ‘continuously available’ and mapped to clients that support SMB3.0.

The feature uses persistent handles, which offer a longer access period to files. The handle is requested with the persistent flag, and the timeout specified is carried out, unlike in the case of durable handles.

SMB Witness - SWN

The Witness service allows the clients on a network to get notified in case a server experiences disruption or latency. In SMB1.0 and to some extent in SMB2.x, this was achieved with a time-out service (based on remote procedure call). This however took a long time (up to 45 seconds). In enterprise networks, a faster notification was needed for a quick failover to another server, and thus the Service Witness Protocol was introduced in the new SMB dialects.

SWN is thus a separate protocol by itself, and notifies clients using SMB3.x of any latencies and disruptions on the server side. SWN is independent of the SMB protocol and runs independently, as part of the Continuously Available framework.

Transparent Failover

With transparent failover, users remain connected to their data if one of the servers breaks down, as another server smoothly kicks into action. The failover function thus allows applications to continue working as usual. Hence the name, ‘transparent failover’.

Transparent failover is a key feature of the 3.x family of SMB as per Microsoft. This feature is of great importance considering the rapid networking speed offered by SMB.

The new protocol supports Hyper-V virtual machines and SQL Server with automatic management of connections across nodes in a clustered environment. These applications are not very adept at handling server failures that can result in offline databases and in cases where a virtual server can no longer access storage and crashes. With SMB3.0 onwards, Windows file sharing is available all the time as transparent failover ensures that storage and data connections stay online.

The new protocol supports Hyper-V virtual machines and SQL Server with automatic management of connections across nodes in a clustered environment. These applications are not very adept at handling server failures that can result in offline databases and in cases where a virtual server can no longer access storage and crashes. With SMB3.0 onwards, Windows file sharing is available all the time as transparent failover ensures that storage and data connections stay online.

  • Windows Server 2012 with at least two nodes on a failover cluster
  • Clearing of the Validate Configuration wizard by servers, storage and network
  • File server role available on all nodes of a cluster
  • Cluster file server set for file shares with continuously available property

For system administrators of clustered file servers and nodes, transparent failover is a great relief, as they can perform maintenance without loss of network connections.

Scale Out File Servers

SOFS is the use of multiple servers (instead of just one, which is liable to fail or crash) so that client applications can access database storage. In case a server is not available, other servers ensure a non-interrupted file access to users. This feature can only be used when both the server and client run on SMB3.x.

This feature is available from Windows Server 2012 onwards, and allows setting up high performance and always available Scale-Out File Servers (SOFS), where user permissions can be easily managed.

Handling enterprise data and workload on storage can now be done on file-based systems, and SMB3.x supports Hyper-V and SQL Server for such tasks. SOFS is the result of this transformation brought about by the latest protocol, with the added advantages of failover and scalability.

In a typical clustered network scenario, SMB3.0 offers parallel data access to all nodes by publishing file shares using version 2 of clustered shared volumes. Clients can also use the totality of resources available in the form of file server nodes and balance load, thus removing bandwidth restrictions found in single cluster nodes.

Clients running SMB1 protocol are denied access when connecting to a scale-out server system. Those running SMB2.0 can connect to SOFS, but cannot take advantage of the transparent failover feature.

Clients running SMB1 protocol are denied access when connecting to a scale-out server system. Those running SMB2.0 can connect to SOFS, but cannot take advantage of the transparent failover feature.


Sources:

Pre-authentication:

https://blogs.msdn.microsoft.com/openspecification/2017/05/26/smb-2-and-smb-3-security-in-windows-10-the-anatomy-of-signing-and-cryptographic-keys/

Durable and persistent handles

https://library.netapp.com/ecmdocs/ECMP1196891/html/GUID-897AB973-A624-4C52-AD90-6928D769F9D3.html

https://serverfault.com/questions/759359/difference-between-durable-file-handles-resilient-file-handles-and-persistent-f

Compounding

https://en.wikipedia.org/wiki/Server_Message_Block https://blogs.technet.microsoft.com/josebda/2008/12/09/smb2-a-complete-redesign-of-the-main-remote-file-protocol-for-windows/

Concurrent

https://support.riverbed.com/bin/support/static/lrj6rq1evg0fnm7pekuq3j2md1/html/vte4p5uj2dkg9k1ukjv82835ci/sh_9.2_dg_protocols_html/index.html#page/sh_9.2_dg_protocols/cifs.html

MTU

https://blogs.msdn.microsoft.com/openspecification/2009/06/22/smb-2-1-multi-credit-large-mtu-operations/

https://www.synology.com/en-global/knowledgebase/DSM/help/DSM/AdminCenter/file_winmacnfs_win

https://docs.microsoft.com/en-us/windows-server/storage/file-server/file-server-smb-overview

Scalability

https://sourcedaddy.com/windows-7/server-message-block-smb-2-0.html

https://www.purestorage.com/uk/resources/type-a/scale_out_file_services_design_guide_with_windows_server2016_and_pure_storage_flasharray.html

Oplock and lease

https://blogs.msdn.microsoft.com/openspecification/2009/05/22/client-caching-features-oplock-vs-lease/

Continuous availability

https://wiki.samba.org/index.php/New_clustering_features_in_SMB3_and_Samba#Continuous_Availability

Witness

http://storagegaga.com/smb-witness-protection-program/

Encryption

https://docs.microsoft.com/en-us/windows-server/storage/file-server/smb-security http://www.admin-magazine.com/Archive/2017/40/SMB-3.1.1-in-Windows-Server-2016

Transparent failover

https://www.derekseaman.com/2012/10/windows-server-2012-smb-transparent.html https://medium.com/the-andela-way/scaling-out-with-node-clusters-1dca4a39a2a

https://blogs.technet.microsoft.com/clausjor/2012/06/07/smb-transparent-failover-making-file-shares-continuously-available/

Scale out

https://www.starwindsoftware.com/blog/highly-available-scale-out-file-server-with-smb3 https://www.starwindsoftware.com/resource-library/starwind-smb3

https://www.youtube.com/watch?v=JpMr9yk_G3Q

Multichannel

https://blogs.technet.microsoft.com/josebda/2012/06/28/the-basics-of-smb-multichannel-a-feature-of-windows-server-2012-and-smb-3-0/

https://level1techs.com/video/smb-multichannel-how-it-works-troubleshooting-guide

SMB Direct

https://virtualizationreview.com/Articles/2015/09/16/Using-SMB-Direct-in-the-Real-World.aspx?Page=2

Directory leasing

https://searchwindowsserver.techtarget.com/tip/SMB-30-brings-fault-tolerant-features-more-speed