Network Block Device (TCP version)

What is it: With this thing compiled into your kernel, Linux can use a remote server as one of its block devices. Every time the client computer wants to read /dev/nd0, it will send a request to the server via TCP, which will reply with the data requested. This can be used for stations with low disk space (or even diskless - if you boot from floppy) to borrow disk space from other computers. Unlike NFS, it is possible to put any file system on it. But (also unlike NFS), if someone has mounted NBD read/write, you must assure that no one else will have it mounted.

Limitations:It is possible, but not recommended, to use NBD as a root file system, as a user-land program is required to start (you can get it done with initrd; Christian Laurs did that, and put some scripts up on his webpage). It also allows you to run a read-only block-device in user-land (making server and client physically the same computer, communicating using loopback). Please note that read-write nbd with client and server on the same machine is a bad idea: expect deadlock within seconds (this may vary between kernel versions, maybe on one sunny day it will be even safe?). More generally, it's a bad idea to create loop in 'rw mounts graph'. I.e., if machineA is using device from machineB readwrite, it's a bad idea to use device on machineB from machineA.

Read-write nbd with client and server on some machine has a rather fundamental problem: when the system is short of memory, it tries to write back dirty pages. So the nbd client asks the nbd server to write back data, but as nbd-server is a userland process, it may require creating dirty pages to fullfill the request. That way lies the deadlock. Note also that the README in the source has a (slightly) more in-depth discussion of this problem.

Current state: It currently works. Network block device seems to be pretty stable. It was originaly thought that it is impossible to swap over TCP. This turned out not to be true, though the patch required to make it safe has not been forward-ported to any reasonably recent kernel.

If you want swapping to work, first make nbd working. (You'll have to mkswap on server; mkswap tries to fsync which will fail.) Now, you have a version which mostly works. Ask me for kreclaimd if you see deadlocks.

Network block device has been included into standard (Linus') kernel tree in 2.1.101.

You can run RAID5 and md over nbd.

Comments and help about the tools is appreciated; if you're willing to help out, please subscribe to the mailinglist, and share your thoughts.

Versioning scheme

Starting around version 2.6, the NBD tools are versioned as follows: <major version number>.<minor version number>.<bugfix version number>

With:

For example, version 2.8.3 means that there were new features since 2.7.x, and that there have been three releases which only fixed bugs since 2.8.0.

Links

NBD on other platforms

Besides Linux, the authors have knowledge of NBD client support on the following platforms:

The server should theoretically work on every POSIX-compliant platform out there (provided glib is supported on that platform). If it doesn't, that's a bug and I want to know about it; if you encounter such problems, please send a mail to the mailinglist.


SourceForge Logo  

This page maintained by Wouter Verhelst.