A Journey
A couple of summers ago, I rode the Mickelson Trail through the Black Hills of South Dakota, USA. It’s a gravel trail, roughly 109 miles (175km) long. I rode my old mountain bike. The northern end starts in the town of Deadwood, and the southern end is in Edgemont, and you will be entirely forgiven if you have never heard of either1.
The ride, at times, was lonely and daunting. There were surprises and setbacks along the way. The trail was rough, the territory unfamiliar, and cell phone coverage was typically nil. I would do it again in a heartbeat given the chance.
It was a lot like other journeys I’ve taken, including that one time when I set out to write an SMB toolkit in Java. Just as I know how to ride a bicycle, I knew enough about SMB. Java, however, was a new and different landscape.
The Backstory
It seemed like a crazy idea at the time, but everything we were doing back then seemed a little crazy.
There were more than a dozen of us, scattered around the world, stealing time between work, studies, and family to analyze network traces and write code. A lot of code. We were building Samba, the Open Source SMB Server suite, and we were doing it in the dark. The only available documentation was years out of date, and it would be several more years before updated specifications would be written and released2. We had to figure it all out for ourselves, with limited resources and the looming threat of reprisal. We were, in those days, the rebel scum3 taking on the evil empire.
and in the midst of all that, I came up with this crazy idea: Let’s do it all over again in Java!
Yeah…how hard could that be?
We’re talking about the final flailing years of the 20th Century. Java was still a bright shiny new thing, and it was being promoted like diet soda. It could be embedded in web pages, run on desktops and servers, and there were a handful of companies producing tiny single board Java devices for tinkering. It seemed like a perfect platform for an SMB engine.
So I set up a project web page, bought my first Java book, and started writing and publishing code. I gave the project the name jCIFS because, back then (following an attempt at rebranding) the SMB protocol was being called CIFS—I’ll explain that episode a little later on.
It took a while, but jCIFS began to attract a few curious onlookers, then a small following, and eventually some contributors. There was one in particular who started off by criticizing the quality of my newbie Java code. He was (annoyingly) quite right, of course, so he took over as project lead (which was plenty fine by me), and my role shifted to providing SMB know-how and being a conduit to the accumulated wisdom of the Samba Team. The crazy idea had taken off and was flying.
That’s probably when I started writing protocol docs. People would ask questions about SMB behavior, and my answers grew from e’mail messages to blog posts to magazine articles. I was invited to help write the SNIA4 SMB Technical Reference, and eventually produced a book about SMB/CIFS internals.
On the Trail with jCIFS
We had no clear idea, when we set out, exactly where we were going. We didn’t know how jCIFS might be used, or why, or by whom. Building it just seemed like the right thing to do, so we did it. Before too long we had hammered jCIFS into fairly decent shape, and we started getting feedback from people who were using it on supercomputers, in media players, and other odd places. There were even a few embedded systems developers tinkering with it. jCIFS quickly became the go-to toolkit for playing with SMB. On our mailing list, we had an endless supply of questions about service discovery, share enumeration, and access controls. We were feeling pretty good about our little project.
Every year, there are a handful of conferences and testing workshops organized for the SMB community. The jCIFS collaboration, unfortunately, was both unfunded and geographically scattered. Since Samba was better known it had better outside support and, since I was on both teams, it fell to me to represent jCIFS at events. These events provided an opportunity to test against a wide variety of implementations, both open source and commercial, and also let me introduce jCIFS to a wider audience.
…and maybe that’s how jCIFS wound up on the radar of Android app developers, or perhaps they just stumbled across it on the web. Whatever. The point is that Android developers discovered that they needed jCIFS, and jCIFS was there for Android.
The relationship wasn’t exactly publicized. The first I knew about it was when I loaded a file browser app on my ‘phone and, out of curiosity and old habit, watched the packet capture as the app connected to my Samba server. The connection request had a tell-tale marker—jCIFS actually tells you that it’s jCIFS. With a little research, I found that there were several apps out there all using jCIFS as their SMB engine.
The Android/jCIFS relationship is still strong today, though there is a bit of strain. jCIFS implements SMB/CIFS (now known as SMB1), which has been supplanted by SMB2 and SMB3. The original SMB/CIFS protocol, which dates back to the early 1980’s, has been deprecated. Newer versions of Windows disable it by default.
SMB Rides the Bumpy Road
I promised I would explain about the “CIFS” name.
The full story is kinda complex, but I’ll try to keep it simple.
SMB was originally named BAF, the initials of the developer who created it for IBM back in the early 80’s, but IBM decided that “Server Message Block” somehow sounded better. It was created for PC-DOS and MS-DOS, later ported to OS/2, and finally found its way into WindowsNT and the Windows9x series.
In the mid-1990’s, Microsoft decided to change the protocol’s name to Common Internet File System, CIFS. The key thing about the new name was that it included the word “Internet”, which was critical for marketing in the mid-1990’s. It was, however, exactly the same protocol.
It was expected, reasonably, that along with the new name Microsoft would provide an updated dialect, adding new capabilities, but that didn’t happen. Not right away, at least. Instead, Microsoft spent the next several years embroiled in a pair of high-profile antitrust lawsuits brought by major world governments. Oddly, during that time, the CIFS name somehow got turned around. Instead of referring to a new protocol version with added features and functionality, it wound up meaning SMB as it had been in Windows NT4 and earlier.
The protocol upgrade that had been expected with Windows2000 did, eventually, arrive. It was hidden as a surprise inside WindowsVista, and it was not called CIFS. It was, instead, called SMB2 and it was much cleaner and faster than its predecessor. A few years later we got SMB3 which added all sorts of speed, reliability, and scaling features. SMB3, however, was not another full protocol overhaul. SMB2 and SMB3 are the same protocol, but SMB3 has more features5. Old SMB/CIFS is now referred to as SMB1 and, as mentioned earlier, Microsoft officially deprecated the original SMB1 protocol a few years back.
Back when jCIFS was created, writing an SMB/CIFS stack was like building an automotive transmission from parts, some of which were missing and, before the official protocol specifications were released, the work was essentially being done in the dark with only a flashlight and some loose matches to see by.
Then things changed. SMB2, SMB3, and a pile of new documentation were all released by Microsoft, greatly simplifying the job of creating a compatible implementation. By that time, however, jCIFS was already fairly well established and developer focus had shifted to new projects and priorities. jCIFS did not take up the challenge of supporting SMB2 and SMB3. It fell to others to pick up the slack. The jCIFS website is still available, as is the code, but there have been few (if any) updates for a couple of years now.
A New Hope
A little bit of web searching shows that the jCIFS code lives on at the heart of newer projects, particularly one called jCIFS-NG6 that is busy filling in the missing SMB2/SMB3 gaps. jCIFS also inspired the jNQ product from Visuality7.
Also in the Open Source arena, Samba is still the gold standard more than twenty years since I first joined the project. People don’t talk about Samba that much anymore. Other projects have taken the spotlight, but that doesn’t mean that there isn’t a lot still going on. The same slightly crazed and tangential thinking that led me to start the jCIFS project has also led to a much more recent project called Zambezi. It got its name because it needed a name and Zambezi was the first slightly crazed and tangential name that came to mind. Zambezi is written in C, rather than Java, and it is aimed at empowering SmartNICs and general-purpose Data Processing Units (DPUs).
SmartNICs are high-end network cards that have on-board processors (network computers on a card). The SmartNICs can do a lot of the up-front protocol work that, historically, the computer itself had to do. Moving SMB protocol stuff to the SmartNIC frees up resources so that everything moves faster. SmartNICs can also offload things like encryption, compression, error handling, and support for multiple transports (TCP, RDMA, and QUIC).
The Road Ahead
That Mickelson Trail ride was a challenge, and I like a good challenge. It’s my kind of fun.
Implementing SMB is also a worthy challenge, but it can be quite a challenge. I know some experienced SMB developers who have managed to pull a team together and get the job done in just a couple of years8. I have also heard many sad stories of failed or aborted attempts. SMB is tricky. You need to know what you’re up against before you set out on this journey. I recommend finding a guide.
Further Reading
Here are some links to help you further enjoy SMB.
- Implementing CIFS – The Common Internet Filesystem An online book about SMB/CIFS and associated protocols.
- jCIFS – The jCIFS project website.
- jCIFS-NG – The jCIFS codebase has been picked up by a new project and is still under active development.
- The Deprecation of SMB1 – “You should be planning to get rid of this old SMB dialect.” A blog post by Jose Barreto, Microsoft, 2015
- The Zambezi – SMB3/SmarktNIC Project As is usual for me, the comments outweigh the code.
1 You may have heard of Deadwood. It was the titular setting of a TV mini-series, and played a role in various bits of US history.
2 I should know, I was handed the job of writing them.
3 In this instance, “scum” stands for Super Coding Uber Monkeys.
4 Storage Networking Industry Association (SNIA)
5 Prior to its release, SMB3 was known as SMB2.2.
6 NG == New Generation? Next Generation? NeoGenesis? I’m not sure.
7 These folks are friends of mine. The SMB developer community is small but vibrant.
8 For comparison, crossing Iowa by bicycle takes about a week. It’s approximately 500mi (800km) from the Missouri River to the Mississippi. A lot of work, but worth the effort.
Christopher Hertel, Sr. Manager of Advanced Computing, UMN
