Linux File System

Operating Systems, the software that powers your computer, rely on a crucial element known as the file system.
In the Linux world, a diverse range of file systems has emerged, each crafted to address specific needs and preferences.
The Linux file system is a multifaceted structure comprised of three essential layers.
At its foundation, the Logical File System serves as the interface between user applications and the file system, managing operations like opening, reading, and closing files.
Above this, the Virtual File System facilitates the concurrent operation of multiple physical file systems, providing a standardized interface for compatibility.
Finally, the Physical File System is responsible for the tangible management and storage of physical memory blocks on the disk, ensuring efficient data allocation and retrieval.
Together, these layers form a cohesive architecture, orchestrating the organized and efficient handling of data in the Linux operating system.

Linux File System Structure:

A file system mainly consists of 3 layers. From top to bottom:

Logical File System:

The Logical File System acts as the interface between the user applications and the file system itself.
It facilitates essential operations such as opening, reading, and closing files.
Essentially, it serves as the user-friendly front-end, ensuring that applications can interact with the file system in a way that aligns with user expectations.

Virtual File System:

The Virtual File System (VFS) is a crucial layer that enables the concurrent operation of multiple instances of physical file systems.
It provides a standardized interface, allowing different file systems to coexist and operate simultaneously.
This layer abstracts the underlying complexities, ensuring compatibility and cohesion between various file system implementations.

Physical File System:

The Physical File System is responsible for the tangible management and storage of physical memory blocks on the disk.
It handles the low-level details of storing and retrieving data, interacting directly with the hardware components.
This layer ensures the efficient allocation and utilization of physical storage resources, contributing to the overall performance and reliability of the file system.

Characteristics of a File System:

Space Management - how the data is stored on a storage device. Pertaining to the memory blocks and fragmentation practices applied in it.
Filename - a file system may have certain restrictions to file names such as the name length, the use of special characters, and case sensitive-ness.
Utilities - file systems provide features for initializing, deleting, renaming, moving, copying, backup, recovery, and control access of files and folders.
Directory - the directories/folders may store files in a linear or hierarchical manner while maintaining an index table of all the files contained in that directory or subdirectory.
Metadata - for each file stored, the file system stores various information about that file’s existence such as its data length, its access permissions,
device type, modified date-time, and other attributes. This is called metadata.

Some important terms:

Journaling - File systems keep a log called the journal, that keeps track of the changes made to a file but
not yet permanently committed to the disk so that in case of a system failure the lost changes can be brought back.
Versioning - File systems store previously saved versions of a file, i.e., the copies of a file are stored based on previous
commits to the disk in a minutely or hourly manner to create a backup.
Inode - The index node is the representation of any file or directory based on the parameters – size, permission, ownership, and location of the file and directory.

Now let's dive into the different ways Linux implements file systems for disk storage devices.

Types Of Linux File Systems:

ext (Extended File System) - Implemented in 1992, it is the first file system specifically designed for Linux. It is the first member of the ext family of file systems.

ext2 - The second ext was developed in 1993. It is a non-journaling file system that is preferred to be used with flash drives and SSDs.
It solved the problems of separate timestamp for access, inode modification and data modification.
Due to not being journaled, it is slow to load at boot time.

Xiafs - Also developed in 1993, this file system was less powerful and functional than ext2 and is no longer in use anywhere.

ext3 - The third ext developed in 1999 is a journaling file system. It is reliable and unlike ext2,
it prevents long delays at system boot if the file system is in an inconsistent state after an unclean shutdown.
Other factors that make it better and different than ext2 are online file system growth and HTree indexing for large directories.

JFS (Journaled File System) - First created by IBM in 1990, the original JFS was taken to open source to be implemented for Linux in 1999.
JFS performs well under different kinds of load but is not commonly used anymore due to the release of ext4 in 2006 which gives better performance.

ReiserFS - It is a journal file system developed in 2001. Despite its earlier issues, it has tail packing as a scheme to reduce internal fragmentation.
It uses a B+ Tree that gives less than linear time in directory lookups and updates.
It was the default file system in SUSE Linux till version 6.4, until switching to ext3 in 2006 for version 10.2.

XFS - A 64-bit journaling file system and was ported to Linux in 2001.
It now acts as the default file system for many Linux distributions.
It provides features like snapshots, online defragmentation, sparse files, variable block sizes, and excellent capacity. It also excels at parallel I/O operations.

SquashFS - Developed in 2002, this file system is read-only and is used only with embedded systems where low overhead is needed.

Reiser4 - It is an incremental model to ReiserFS. It was developed in 2004. However, it is not widely adapted or supported on many Linux distributions.

ext4 - The fourth ext developed in 2006, is a journaling file system.
It has backward compatibility with ext3 and ext2 and it provides several other features, some of which are persistent pre-allocation,
unlimited number of subdirectories, metadata checksumming and large file size.
ext4 is the default file system for many Linux distributions and also has compatibility with Windows and Macintosh.

btrfs (Better/Butter/B-tree FS) - It was developed in 2007.
It provides many features such as snapshotting, drive pooling, data scrubbing, self-healing and online defragmentation.
It is the default file system for Fedora Workstation.

bcachefs - This is a copy-on-write file system that was first announced in 2015 with the goal of performing better than btrfs and ext4.
Its features include full filesystem encryption, native compression, snapshots, and 64-bit check summing.

Others - Linux also has support for file systems of operating systems such as NTFS and exFAT, but these do not support standard Unix permission settings.
They are mostly used for interoperability with other operating systems.

The table below lists out the criteria on which filesystems can be compared, Please note that there are more criteria than the ones listed in the table.
This table is supposed to give you an idea of how file systems have evolved.

Linux Directories:

/bin

Directory that contains binaries, that is, some of the applications and programs you can run.
You will find the ls program mentioned above in this directory, as well as other basic tools for making and removing files and directories, moving them around, and so on.
There are more bin directories in other parts of the file system tree, but we’ll be talking about those in a minute.

/boot

Directory contains files required for starting your system. Do I have to say this? Okay, I’ll say it: DO NOT TOUCH!!!
If you mess up one of the files in here, you may not be able to run your Linux and it is a pain to repair.
On the other hand, don’t worry too much about destroying your system by accident: you have to have superuser privileges to do that.

/dev

Contains device files. Many of these are generated at boot time or even on the fly.
For example, if you plug in a new webcam or a USB pendrive into your machine, a new device entry will automagically pop up here.

/etc

Directory where names start to get confusing. /etc gets its name from the earliest Unixes and it was literally “et cetera”
because it was the dumping ground for system files administrators were not sure where else to put.
Nowadays, it would be more appropriate to say that etc stands for “Everything to configure,” as it contains most, if not all system-wide configuration files.
For example, the files that contain the name of your system, the users and their passwords, the names of machines on your network and when and where the partitions on your hard disks should be mounted are all in here. Again, if you are new to Linux, it may be best if you don’t touch too much in here until you have a better understanding of how things work.

/home

Where you will find your users personal directories.
In my case, under /home there are two directories: /home/adam, which contains all my stuff; and /home/guest, in case anybody needs to borrow my computer.

/lib

Where libraries live. Libraries are files containing code that your applications can use.
They contain snippets of code that applications use to draw windows on your desktop, control peripherals, or send files to your hard disk.
There are more lib directories scattered around the file system, but this one, the one hanging directly off of / is special in that, among other things,
it contains the all-important kernel modules. The kernel modules are drivers that make things like your video card, sound card, WiFi, printer, and so on, work.

/media

Where external storage will be automatically mounted when you plug it in and try to access it.
As opposed to most of the other items on this list, /media does not hail back to 1970s,
mainly because inserting and detecting storage [pendrives, USB hard disks, SD cards, external SSDs, etc] on the fly, while a computer is running, is a relatively new thing.

/mnt

Directory, however, is a bit of remnant from days gone by. This is where you would manually mount storage devices or partitions. It is not used very often nowadays.

/opt

Directory is often where software you compile (that is, you build yourself from source code and do not install from your distribution repositories) sometimes lands.
Applications will end up in the /opt/bin directory and libraries in the /opt/lib directory.
A slight digression: another place where applications and libraries end up in is /usr/local, When software gets installed here,
there will also be /usr/local/bin and /usr/local/lib directories.
What determines which software goes where is how the developers have configured the files that control the compilation and installation process.

/proc

Like /dev is a virtual directory. It contains information about your computer, such as information about your CPU and the kernel your Linux system is running.
As with /dev, the files and directories are generated when your computer starts, or on the fly, as your system is running and things change.

/root

The home directory of the superuser [also known as the “Administrator”] of the system.
It is separate from the rest of the users’ home directories BECAUSE YOU ARE NOT MEANT TO TOUCH IT.
Keep your own stuff in your own directories, people.

/run

Another new directory. System processes use it to store temporary data for their own nefarious reasons. This is another one of those DO NOT TOUCH folders.

/sbin

Similar to /bin, but it contains applications that only the superuser [hence the initial s] will need.
You can use these applications with the sudo command that temporarily concedes you superuser powers on many distributions.
/sbin typically contains tools that can install stuff, delete stuff and format stuff.
As you can imagine, some of these instructions are lethal if you use them improperly, so handle with care.

/usr

Directory was where users home directories were originally kept back in the early days of UNIX.
However, now /home is where users kept their stuff as we saw above. These days, /usr contains a mish-mash of directories which in turn contain applications,
libraries, documentation, wallpapers, icons and a long list of other stuff that need to be shared by applications and services.
You will also find bin, sbin and lib directories in /usr. What is the difference with their root-hanging cousins? Not much nowadays.
Originally, the /bin directory (hanging off of root) would contain very basic commands, like ls, mv and rm,
the kind of commands that would come pre-installed in all UNIX/Linux installations, the bare minimum to run and maintain a system.
/usr/bin on the other hand would contain stuff the users would install and run to use the system as a work station, things like word processors, web browsers, and other apps.
But many modern Linux distributions just put everything into /usr/bin and have /bin point to /usr/bin just in case erasing it completely would break something.
So, while Debian, Ubuntu and Mint still keep /bin and /usr/bin (and /sbin and /usr/sbin) separate, others,
like Arch and its derivatives just have one “real” directory for binaries, /usr/bin, and the rest or bins are “fake” directories that point to /usr/bin.

/srv

Directory contains data for servers. If you are running a web server from your Linux box, your HTML files for your sites would go into /srv/http (or /srv/www).
If you were running an FTP server, your files would go into /srv/ftp.

/sys

Another virtual directory like /proc and /dev and also contains information from devices connected to your computer.
In some cases you can also manipulate those devices. For example, change the brightness of the screen of laptop by modifying the value stored in the
/sys/devices/pci0000:00/0000:00:02.0/drm/card1/card1-eDP-1/intel_backlight/brightness file [on your machine you will probably have a different file].
But to do that you have to become superuser.
The reason for that is, as with so many other virtual directories, messing with the contents and files in /sys can be dangerous and you can trash your system.
DO NOT TOUCH until you are sure you know what you are doing.

/tmp

Contains temporary files, usually placed there by applications that you are running.
The files and directories often [not always] contain data that an application doesn’t need right now, but may need later on.
You can also use /tmp to store your own temporary files — /tmp is one of the few directories hanging off / that you can actually interact with without becoming superuser.

/var

Was originally given its name because its contents was deemed variable, in that it changed frequently.
Today it is a bit of a misnomer because there are many other directories that also contain data that changes frequently, especially the virtual directories we saw above.
Be that as it may, /var contains things like logs in the /var/log subdirectories. Logs are files that register events that happen on the system.
If something fails in the kernel, it will be logged in a file in /var/log; if someone tries to break into your computer from outside, your firewall will also log the attempt here.
It also contains spools for tasks. These “tasks” can be the jobs you send to a shared printer when you have to wait because another user is printing a long document,
or mail that is waiting to be delivered to users on the system. Your system may have some more directories we haven’t mentioned above.
In the screenshot, for example, there is a /snap directory. That’s because the shot was captured on an Ubuntu system.
Ubuntu has recently incorporated snap packages as a way of distributing software. The /snap directory contains all the files and the software installed from snaps.