Directories, and the Shapes They Take
Chapter One Hundred One
Syllabus topic Module 2, "File System Interface - Directory and Disk Structure"
Pages 406 to 411 of 452
In one line
A directory is a file whose contents are a table of names and the file identifiers they stand for, and the interesting question is what shape the collection of directories has.
What a directory is
A directory is a symbol table: it translates a name into the information the file system needs about a file. And it is itself a file, stored in blocks like any other, which is why Chapter one hundred six is about how to search one efficiently.
| Operation on a directory | What it must do |
|---|---|
| search for a name | the operation everything else is built on |
| create a file | add a name and its identifier |
| delete a file | remove the entry |
| list the directory | print the names, and usually some attributes |
| rename a file | change the name, keeping the identifier |
| traverse the file system | visit every directory and every file, for a backup or a search |
Shape one: a single level directory
One directory, for everybody and everything.
| Simple | one table, and the search is obvious |
| names must be unique across the whole machine | two users cannot both have a file called notes |
| no grouping | a user with forty files sees forty files |
The name collision is fatal on any machine with more than one user, and it was fatal even on single user machines as soon as the disk was large enough to hold a few hundred files.
Shape two: a two level directory
One directory per user, and a master directory that names the users.
| Name | What it is |
|---|---|
| the master file directory, MFD | one entry per user, pointing at that user's directory |
| a user file directory, UFD | that user's files |
A path is then the user name and the file name, and two users may both have notes because the path distinguishes them.
| Solves the collision problem between users | |
| no grouping within a user | one user's forty files are still forty files in one list |
| system files are a problem | every user would need a copy of the compiler, unless the system looks in a special directory too: a search path |
That search path is the ancestor of the PATH variable of Chapter nine of Module 1, and it exists for exactly this reason.
Shape three: a tree structured directory
A directory may contain directories, to any depth. This is what every system has used for fifty years.
| Idea | What it means |
|---|---|
| root | the directory every path starts from |
| absolute path name | the path from the root, for example /home/student/notes/os.txt |
| relative path name | the path from the current working directory, for example notes/os.txt |
| current directory | one per process, changed by cd, and inherited by a child |
| the bit that says directory | in the entry, so the system knows which entries can be descended into |
Directories, and the Shapes They Take
Deleting a directory needs a policy, and a question asks for both options. Either a directory must be empty before it can be deleted, and the user clears it first, or the system deletes everything inside it recursively, which is convenient and dangerous. Systems offer both, under different names.
Shape four: an acyclic graph directory
Let two directories share one file, or one subdirectory, so the structure is no longer a tree. Two programmers on one project need the same file to be in both their directories, and a copy is not the same thing: a copy does not change when the other person edits it.
| Way to share | How it works |
|---|---|
| a hard link: a second directory entry naming the same file identifier | both names are the file; neither is the original |
| a symbolic link: a small file holding a path | the name is a pointer to a path, resolved when it is used |
| a duplicate entry copying the file's details | never done: the two copies of the attributes get out of step |
The three problems, and this is what a question asks for
| Problem | What goes wrong | The answer |
|---|---|---|
| several absolute path names for one file | a traversal visits the same file twice, so a backup copies it twice | the traversal must recognise it, usually by identifier |
| deletion | removing one name must not leave the other pointing at nothing | keep a reference count in the file, and free the file when it reaches zero |
| a symbolic link left dangling | the target is deleted and the link still exists | accept it: using the link fails, and the link may be deleted or left |
The reference count is the answer to the deletion problem, and it is the same counter as the open count of Chapter ninety nine and the shared frame count of Chapter seventy seven. The idea appears three times in this module for the same reason: something is shared, and the last user must be the one who frees it.
Demonstrated on the lab machine
$ printf 'the contents\n' > report.txt
$ ln report.txt copy-of-report.txt
$ stat -c '%n has %h name(s) and inode %i' report.txt copy-of-report.txt | sed 's/inode [0-9]*/inode N/'
report.txt has 2 name(s) and inode N
copy-of-report.txt has 2 name(s) and inode N
$ stat -c %i report.txt copy-of-report.txt | sort -u | wc -l
1
$ ln -s report.txt shortcut.txt
$ stat -c '%n is a %F of %s bytes' shortcut.txt
shortcut.txt is a symbolic link of 10 bytes
$ rm report.txt
$ echo "after removing the first name, the second still reads:"; cat copy-of-report.txt
after removing the first name, the second still reads:
the contents
$ stat -c '%n now has %h name(s)' copy-of-report.txt
copy-of-report.txt now has 1 name(s)
$ cat shortcut.txt
cat: shortcut.txt: No such file or directoryDirectories, and the Shapes They Take
Every line of the theory above is in that transcript.
| The machine says | What it proves |
|---|---|
| both names report 2 names and the inode numbers reduce to one | a hard link is not a copy: one file, two names, and the reference count is the 2 |
| the symbolic link is a symbolic link of 10 bytes | it is a file of its own, holding the ten characters report.txt and nothing else |
after rm report.txt the other name still reads the contents, and the count is now 1 | rm removed a name and decremented the count; the file is freed only at zero |
cat shortcut.txt fails | the symbolic link is now dangling: it holds a path that no longer resolves |
The link count of a directory
The cleanest proof that . and .. are ordinary entries.
$ mkdir project
$ stat -c 'a new directory has a link count of %h' project
a new directory has a link count of 2
$ mkdir project/chapter1 project/chapter2
$ stat -c 'with two subdirectories it has %h' project
with two subdirectories it has 4
$ ls -a project
. .. chapter1 chapter2An empty directory has two links, not one. Its name in the parent is one, and its own . entry is the other. Each subdirectory then adds one more, because each holds a .. entry pointing back. Two subdirectories therefore make four, and the arithmetic is exact:
links = 1 + 1 + 2 = 4
So a directory's link count is 2 plus the number of subdirectories, which is a fact worth knowing: it tells you how many subdirectories a directory has without listing it.
Shape five: a general graph directory
Allow a link from a directory to one of its own ancestors, and the structure has a cycle.
| Problem a cycle causes | Why it is serious |
|---|---|
| a traversal never ends | searching for a file can go round the cycle for ever |
| reference counts never reach zero | a cycle's entries refer to each other, so the count stays positive after the last real name is gone, and the space is never freed |
| The way out | What it costs |
|---|---|
| allow links only to files, never to directories | the structure stays acyclic and the reference count works |
| limit the number of links a traversal will follow | simple, and it makes a legal deep structure fail |
| run a cycle detection algorithm when a link is added | expensive, and it must run on every link |
| garbage collection: traverse everything, mark what is reachable, then free the rest | correct, and it is extremely slow on a large disk, so it is done rarely |
Directories, and the Shapes They Take
UNIX takes the first way out: a hard link to a directory is not allowed, so the graph of hard links is acyclic and the link count can be trusted. Symbolic links can make cycles, and the answer there is the second way out: the system refuses after following a fixed number of links.
Distinctions that carry marks
| Hard link | Symbolic link | |
|---|---|---|
| What it is | a directory entry naming the same file identifier | a file holding a path |
| Uses disk space of its own | no, only the entry | yes: the path is its contents, 10 bytes above |
| Counts towards the link count | yes | no |
| Survives deletion of the other name | yes: the file lives while the count is positive | no: it dangles |
| Can point at another file system | no | yes |
| Can point at a directory | not on UNIX | yes |
| Single level | Two level | Tree | |
|---|---|---|---|
| Name collisions | between all users | between one user's own files | avoided by the path |
| Grouping | none | per user only | to any depth |
| System files | in the same directory | need a search path | an ordinary directory |
| Absolute path | Relative path | |
|---|---|---|
| Starts from | the root | the current working directory |
| Same meaning in every process | yes | no |
What it does not mean
A directory is not a list of files. It is a list of names and identifiers; the files are elsewhere.
A hard link is not a copy. There is one file. Editing through either name changes the same bytes.
Deleting a name is not deleting a file. The file goes when the reference count reaches zero.
A symbolic link is not free. It is a file with its own inode and its own block, holding the path.
A general graph is not a richer tree. It is a structure whose traversal may not end and whose space may never be freed, which is why systems prevent it.
Quick revision
- A directory is a symbol table from names to file identifiers, and it is itself a file.
- Operations: search, create, delete, list, rename, traverse.
- Single level: one directory, so names must be unique machine wide and there is no
grouping.
- Two level: a master file directory of users, each with a user file directory.
Solves collisions between users; needs a search path for system files.
- Tree: directories inside directories, with a root, absolute and relative paths,
and a current directory per process. Deleting a directory: empty only, or recursive.
Directories, and the Shapes They Take
- Acyclic graph: sharing by hard links or symbolic links. Three problems:
several path names, deletion, and dangling links. Deletion is solved by a reference count freed at zero.
- Measured: a hard link gives one inode with two names; a symbolic link is a 10 byte file
holding the path; after removing one name the count is 1 and the contents survive; the symbolic link then dangles.
- A directory's link count is 2 plus its number of subdirectories: an empty one has 2,
and with two subdirectories 1 + 1 + 2 = 4.
- General graph: cycles make traversals endless and reference counts never zero. Ways out:
links to files only, a link limit, cycle detection, or garbage collection. UNIX forbids hard links to directories.
Test yourself
- What is a directory, and what is it stored as? A symbol table translating file names into
the identifiers the file system uses, stored as a file like any other.
- Why is a single level directory unusable? Every name must be unique across the whole
machine and there is no way to group files.
- What does a two level directory solve, and what does it still lack? It stops users
colliding over names by giving each a directory under a master file directory; it still offers no grouping within a user, and system files need a search path.
- Distinguish an absolute path from a relative path. An absolute path starts at the root and
means the same thing in every process; a relative path starts at the process's current working directory.
- Give the two policies for deleting a non empty directory. Refuse unless it is empty, or
delete its contents recursively.
- Name the three problems of an acyclic graph directory and the answer to the second.
Several absolute path names for one file, deletion, and dangling links; deletion is handled by a reference count in the file, freed only when it reaches zero.
- How do a hard link and a symbolic link differ? A hard link is another directory entry for
the same file and counts towards its link count; a symbolic link is a separate small file holding a path, does not count, and dangles if the target goes. Only the symbolic link can cross file systems or point at a directory.
- A directory has a link count of 5. How many subdirectories does it have? Three: the count
is 2 plus the number of subdirectories, one for its name in the parent, one for its own dot entry, and one for each child's dot dot.
- Why are cycles in a directory structure dangerous, and what does UNIX do about it? A
Directories, and the Shapes They Take
traversal may never end and reference counts may never reach zero, so space is never freed; UNIX forbids hard links to directories and limits how many symbolic links it will follow.
The rest of this subject
These notes are cut from the University's printed syllabus. Open the syllabus itself, or the past papers, for the same subject.