I used to be really dependent on autofs for this kind of thing. Back in the day, if you wanted your floppy disk to magically appear, you had to mount it either explicitly (ick), implicitly (fstab), or magically. You'd type "ls /plug/floppy" and some magic would go out and fish the correct mount options.
The magic is still handy for all the cases when you have a file somewhere, but your computer doesn't know it. In my case, that's mostly when a file is on a remote server or on an image file.
To solve the problem, there is autofs. That's a tool that magically knows where to put things because you (or someone else for you) told it to.
The autofs paradigm has two major items:
- mount points - which are directories that autofs watches carefully for access
- maps - which are the magic things that tell autofs how to map things
scp user@marco.example.org:/home/user/example.txt .Wouldn't it be much better for a hundred different reasons if you could just copy the file, say like this:
cp /net/marco.example.org/home/user/example.txt .Why is that better? Well, so far it isn't much. But if you could access the file like that, you could convince your editor to edit the file in place, saving it right where it is, without having to copy it locally. You could, for instance, just add the file path of your build server, and suddenly you run software remotely.
Another, more immediately useful example, is the mounting of .iso files. When you download a Linux (or Debian, or whatever) CD/DVD, it comes as a single file you have to burn to a physical medium. Now, it would be great if you could just mount that file. Wait, you can! Not only can you do that, you can automagically do that.
So, this is what this article is all about. We will see how to use a remote server and an ISO file just like regular directories - and nothing is standing in the way of more and more interesting additions to our file systems.
[Setup]
autofs is well integrated intu Ubuntu: just type the trusted
sudo apt-get install autofsand you are on your way. The dependencies are downloaded, the files installed, the autofs daemon started, and you are ready to go.
So, what happened? Absolutely nothing. By default, the autofs installation does nothing at all. But it has a few tricks in store.
All the action you care about happens in the /etc directory. There, you'll find a messy set of files called auto.[something]. In the current package, there are the following:
- auto.master - the main file
- auto.misc - an example with different hard-coded devices
- auto.smb - another example for SMB (Windows) shared drives
/some/path /etc/auto.something --options/some/path is a random directory you choose that henceforth is where autofs is going to put your shares. Whenever you try to access a subdirectory of /some/path that doesn't already exist, autofs will try to mount it looking up the file /etc/auto.something for a hint at what to do. The --options are, well, optional and help determining the behavior of the automounter. --timeout, for instance, is very common: it tells the automounter to drop a directory if it hasn't been used in a set amount of time.
Of the examples, auto.misc is the simpler one, so we'll look at it first. It is composed of lines made up of three sections, just like auto.master. The sections are different in order (don't ask), but they are logically similar:
key options locationThe "key" argument here is not a path but a name - it's the name of the subdirectory you are trying to access. Say you went with the default in auto.master and assigned the mount /misc to auto.misc, and there is a key "floppy" in there, then whenever you try to access the directory /misc/floppy, the automounter would look at the line starting with "floppy" to decide what to mount (the location) and how (the options).
Here you see that autofs was born to mount network shares, since anything that isn't a straight NFS share (you don't care what that is) requires special handling.
The other file, auto.smb, is a lot more complicated. First of all, it doesn't have any lines like auto.misc. Instead, it is a script you can run on the command line. What it does (if you feel inclined to read the source, follow along) is to use the command "smbclient" to query the network and find shares to mount. Once it found one that matches your desired key, it "mounts" it.
It actually doesn't do the mounting itself - it simply returns a line like the ones in auto.misc (minus the key, since that's a given), thus telling the automounter what to do.
[SSH Mount]
So, now that we looked at the general setup, let's look more closely at mounting SSH servers. From now on, I'll use the convention that the user dude on the machine kde is trying to access a server named marco.example.org using the account guy.
The autofs setup is really simple. We need a file, say auto.ssh, that contains the automounter lines:
marco.example.org -fstype=fuse,allow_other,reconnect,uid=1000,gid=1000 sshfs\#guy@marco.example.org:/Wait, wait, wait!!! What does that all mean? First, you have the key, which we already knew about. Then we have options, which we need to explain and chance. Finally, we have the weirdest thing as location.
Ok, there is such a thing as a filesystem on ssh. We need to install that if we want to access the functionality. That's done easily:
sudo apt-get install sshfs
Now we have the fstype installed, and the weird syntax for the location is explained. The backslash ('\') before the pound sign took me hours to figure out, so I am expecting friendly comments and PayPal donation for the time I saved you.
What about the options? fstype is fuse, which is short for Filesystem in User land (you don't need to know right now - but it's a wonderful project). Reconnect and allow_others are for ease of use. The first one reconnects if the network goes down, the second one allows others that have access to the mount point to access your ssh share (you may want to rethink it on multi-user systems - but if you want ot make sure that your mount is accessible by daemons behind your software, you better set it).
The user and group ID are those of the connecting user, dude. You can find both very easily by typing id on the command line (they are the first two numbers on the result). The first user and group created on a Kubuntu machine have the ids 1000, so that's fairly common.
[Problems]
Yeah. It won't work like that. You have to do some extra work for the setup. (Again, donations appreciated...). First, we need to be able to connect to the server using SSH:
ssh guy@marco.example.orgFirst time we do that, we are asked whether we'd like to accept the key. That's very important (insert security blabla), but in our case, it's mostly a nuisance. You have to connect manually the first time, because otherwise autofs is confused by the reply. So do that.
The next problem is a very immediate security problem. When automount is running, it is using the root account. So it's the root account that need to be able to access the remote server. But the root account doesn't have the credentials, dude has them. What to do?
Well, the easiest thing to do is to give the credentials to root. I mean, if you are root, you have access to the credentials, anyway, so it's quite pointless to hide them. Easiest way to do that is by creating a link between dude's credentials and root's. On a single-user system you could link the entire SSH directory:
sudo ln -s ~dude/.ssh ~root(Assuming root doesn't have a .ssh directory yet.)
Next thing is that autofs can't ask for a password or passphrase directly. We could either use an SSH agent (which I won't cover because it's complicated) or we can use a private key without a passphrase (which is ugly because it's totally insecure).
Now, I can't stress this enough: using a private key without a passphrase is seriously dangerous: it allows anyone with access to the machine with the private key to access all the machines that allow access with it. This setup is mostly for people that, like me, prefer SSH over SMB at home and whose main laptop contains all the valuable information. The idea here is that if someone compromises my laptop, the worst thing already happened. That they can access the backup server from there, not a big deal.
Ok, now that you have given root a passphrase-less key and made it a default key, or loaded an ssh-agent with which the automounter communicates, we are ready to go. Well, we have first to tell the automounter where to put the SSH servers. For that purpose, we add a single line to the file auto.master:
/share/ssh auto.sshThat means that from now on, our servers are going to be mounted under /share/ssh (make sure the directory exists, is owned by root, and has the permissions 755). Restart the autofs daemon with sudo /etc/init.d/autofs restart, and there you go! Now try:
ls /share/ssh/marco.example.organd you will get the directory listing for the root directory of that server. You will be able to see, modify, and create files exactly like the user guy on that server could.
[Loop]
If you are a real geek, you probably have a ton of files that can be mounted as drives. You might have the .iso files you burnt to install Linux, you might have the virtual hard drive of a virtual machine (like VirtualBox or VMWare, or User-Mode-Linux for the courageous).
Mounting those drives is easy, but a real nuisance. Essentially, you type:
mount -o loop filename mountpointand then you can access the file as if you were in the mount point. Say, you have the Karmic CD stored as /home/dude/Downloads/kubuntu-9.10-desktop-i386.iso. To put it onto the directory /share/iso/kubuntu-9.10 you would type:
mount -o loop /home/dude/Downloads/kubuntu-9.10-desktop-i386.iso /share/iso/kubuntu-9.10Of course, the directory usually doesn't exist, so you just mount it as /mnt, and you manage until you need a different version and unmount the first one. It gets messy after a short while.
[The Solution]
What about if you get rid of the haphazard and use a script instead? Why a script, you ask. Well, the problem is that while you may know where the file is, unless you are particularly orderly, your computer won't know where to look. So, instead, we use the utility locate to find the file we need and then mount it to a well-known point.
Assume you want to mount the kubuntu ISO file. First, use locate to find it:
locate kubuntu
You get back a list of files that all match the name kubuntu. If you add the option -b, then you get only files who actual file name matches kubuntu (no files contained in directories named kubuntu. Still, we might have a ton of files.
What do we do? We look them all up and determine whether we can mount them. To do that, we use the tool file, which gives us a guess as to the content of the file. If it is a line that contains the words filesystem data, we know we can mount it and just need to pass the correct filesystem type to mount.
What we will do is prioritize the file names by length first. The shorter file name that matches our string is a better match. Then we look at numbers and select the one with the lower number, if the length is the same. Actually, you can decide to resolve conflicts whichever way you want. You can also decide not to resolve a conflict, and to return an error, in which case the mount fails (I don't like that behavior, even though it might be better, because I am doing this just out of convenience, after all!)
I wrote a Tcl script to do the job for me. I also loaded all the ISO images of CD-Rs I own onto my trusted server, backup, which I access (you guessed it) using SSH. Autofs is configured for loop on backup, and for ssh on this computer. So when I need to find a particular picture from one of the CDs I burnt, I simply type:
and off I go. I don't need to get to the server backup and do the mount manually, and I don't have to care what the name of the ISO file is and where I stored it (in the backup tree or in the live image tree). I don't have to invoke ssh, don't have to worry about scp, and don't have to figure out what to cache on my machine before I fire up my image manipulation program.gwenview /share/ssh/backup/loop/pics-2003
[More Fun]
Autofs is best friends with fuse, the file system in user space. Since fuse development is tons easier and less dangerous than development of file systems for the kernel, there are tons of available options.
If you don't believe me, just type in apt-cache search fuse into a terminal window, and look how many of the hits you can get for a file system. There are exciting things like flickrfs (which mounts your flickr photos), unionfs (which joins two directories together, for instance if you have no room left in one...), glusterfs (for clustered servers), and for everybody annoyed enough with UpPeRcase on UNIX, ciopfs (case insensitive on purpose).
[Update: if you want the Tcl script or a Python version of the same, please let me know.]
No comments:
Post a Comment