The Basics of Linux File Management

Knowing how to navigate within the Linux operating system and work with files and the file system is a critical skill for DevOps engineers. Having that basic knowledge is a fundamental requirement for working effectively with any Linux-based system. If you cannot complete basic tasks, such as copying or moving files, creating directories, and viewing the contents of directories, then you will lose your bearings quickly. So, before going further into the world of Linux, the following are a few basic pointers on how to work with its file and directory structures.

Creating, Moving, Copying, Removing and Finding Files

Before making copies of file or moving them around, you need some to work with. The simplest way to create a file in Linux is to “touch” it into existence. You can use the touch command to create an empty file by adding the file location and name after it. Creating an empty directory is just as easy, using the mkdir (make directory) command instead of touch. To view your newly created file and directory, you can use the ls (list) command. This command has several useful parameters, the most common being -l which stands for a long list. Invoking that command produces a more detailed view of the files and directories in alphabetical order. To move a file or directory to a different location, use the mv command, which can also be used to rename files and directories. To create a copy of a file or directory, the cp command is best. If you need to remove old or otherwise unwanted files, the rm (remove) enables you to do that quickly and easily.

Examples:

touch /home/user/myfirstfile		# this creates a file named myfirstfile into the /home/user directory
mkdir /myfirstdirectory		# this creates a directory named myfirstdirectory into the / directory 
ls -l /home/user			# shows the detailed contents of the /home/user directory 
ls -d /myfirstdirectory 	# shows the /myfirstdirectory (not its contents)
mv /home/user/myfirstfile /opt/myfirstfile	# this moves the file from the /home/user directory into the /opt directory
mv /home/user/myfirstfile /home/user/myfirstfilerenamed	# this renames the /home/user/myfirstfile to /home/user/myfirstfilerenamed
cp /home/user/myfirstfile /opt/	# this creates a copy of /home/user/myfirstfile into the /opt directory
rm /home/user/myfirstfile 		# this removes a file named myfirstfile

The powerful find command can be used to locate any file or directory from anywhere in the file system. Note that it doesn’t matter whether the search string is in the beginning, middle or end of the file name if you add the * and ' symbols to the name argument. In addition, the find command can also be used to find and send other commands to located files by adding the -exec parameter to it. Another search-related command is grep which searches for patterns within the contents of files.

Examples:

find / -name '*searchstring*' 			# this searches the file system for a file which includes "searchstring" in its name
find / -name '*searchstring*' -exec rm {} \; 	# this searches the file system for a file which includes "searchstring" in its name and deletes it with the rm command, the backslash/semicolon symbolizes the end of -exec section
grep searchstring /var/myfirstfile		# this searches for the pattern 'searchstring' from within /var/myfirstfile

File Permissions

Depending on their contents, some files require restricted access. This is where file permissions come into play. There are three basic types of permissions that can be granted depending on whether the item being accessed is a file or a directory. Those permissions are read, write and execute.

File permissions are used for the following purposes:

• Read permissions allow viewing the contents of the file.
• Write permissions allow modifying or deleting the file.
• Execute permissions allow execution of the file as a script or binary.

Directory permissions are used as follows:

• Read permissions allow listing the contents of the folder using the ls command.
• Write permissions allow creation of new files or subfolders, and modification or deletion of existing files or folders.
• Execute permissions allow the affected user to enter the directory, and access files and directories inside.

File and directory permissions can be viewed with the ls -l command where user permissions are divided into three sections in the following format -rwxrwxrwx, where…

• The first position is reserved for file or directory indication (- for regular file, d for directory).
• The second, third and fourth position (rwx) is reserved for the owner user.
• The fifth, sixth and seventh position (rwx) is reserved for the owner group.
• The eighth, ninth and tenth position (rwx) is reserved for anyone other than the owning user or group.

These permissions can be set by using the chmod command. Two separate formats are available: numerical and alphabetical. In the numerical format, each permission has an assigned number. Adding those numbers results in a sum that identifies the correct user, group, and others’ permissions.

• Read permission is equivalent to number 4.
• Write permission is equivalent to number 2.
• Execute permission is equivalent to number 1.

In order to apply new permissions, you need to calculate the read, write and execute permissions for all three interest groups. The same can be done with the alphabetical format, where permissions are assigned to using the same letters that the permissions and interest groups have. Adding permissions is done using the chmod command. Using the numerical format does not take into account which permissions existed before the command, but the alphabetical format does reflect that. The numerical format is a fixed permission setting and the alphabetical option is flexible.

Numerical format explanation

A setUID bit can be set only on a file and when the file is executed, the resulting process will run as the user that owns the file and not as the user who started the program. This means that if I have a file that can be executed as the root user, then I as an ordinary user will also be able to execute the same file in root permissions. This special permission is not widely used due to security reasons. However, one specific place where it is used is the /usr/bin/passwd file which allows all users to change their passwords without root permissions.

A setGID bit can be set on both files and directories, although it is more commonly used with directories. When the setGID bit is added to a file, all the system users can execute the file in the group permissions. However, if the same permission is set to a directory, all the files created in that directory automatically get the parent directory’s group ownership. That means all the files created in that directory become automatically accessible to all the users who are within the parent directory’s group.

All the special permissions are set with the chmod command in both numerical and alphabetical format.

• A Sticky bit is marked with the letter t in the others section and can be assigned as o+t or as number 1 in the first position before ordinary permissions (1xxx).
• A SetUID bit is marked with the letter s in the user section and can be assigned as u+s or as number 4 in the first position before ordinary permissions (4xxx).
• A SetGID bit is also marked with the letter s in the group section and can be assigned as g+s or as number 2 in the first position before ordinary permissions (2xxx).

Examples:

chmod o+t /myfirstdirectory 	# Sets a sticky bit to the directory (alphabetical)
chmod 1755 /myfirstdirectory 	# Sets a sticky bit to the directory (numerical)
chmod g+s /myfirstdirectory  	# Sets a setGID on the directory

The find command can also be used in conjunction with the -perm parameter to determine if there is a setUID and sticky bit type of special permissions in the server.

Examples:

find / -perm /1000 	# find sticky bit files
find / -perm /2000 	# find setGID bit files
find / -perm /4000 	# find setUID bit files

File Ownership

Every file has an owner and a group assigned to it. By using the already familiar ls -l command you can find out the owner and group of a file. The owner is usually the one who created the file and can have almost absolute power over it, depending on the permissions. The group has separate permissions as it can consist of multiple users. All the users belonging to the same group have the same permissions to a file. Changing the owner or group of a file can be easily accomplished with the chown and chgrp commands.

Examples:

chown john /var/myfirstfile		# sets the user john as the file owner
chown john:johnsons /var/myfirstfile 	# sets the user john as the file owner and the group johnsons as the group
chgrp johnsons /var/myfirstfile 	# sets the group johnsons as the group

Compressing Files With TAR

Regardless of the type of machine – a large server or small PC, disk space is a limited commodity. To minimize disk space usage or the time needed to move files from one location to another (especially over the internet), file compression comes into play. A compressed file is essentially an archive that contains one or more files that have been reduced in size. A common tool for compressing files is TAR. The name is derived from “Tape Archive” because in years past, it was used to compress files to be stored on magnetic tape. tar has several parameters that can be difficult to remember:

• -c is used to create an archive.
• -x is used to extract from an archive.
• -z is used to filter the archive through gzip (if this parameter is not used, then the files are not compressed but just copied to an archive).
• -v is used to enable verbose output which lists all the files which are processed.
• -f to use archive file or device ARCHIVE.

There are many more parameters for TAR of course (all visible in the man page of the command), but those listed here are the most commonly used.

Examples:

tar -czvf myfirst.tar.gz /var/myfirstdirectory 	# compress the /var/myfirstdirectory directory into the archive named myfirst.tar.gz
tar -xzvf myfirst.tar.gz 			# extract the archive named myfirst.tar.gz

Conclusion

Knowing how to manage and work with files and directories is a first step into the world of Linux CLI, and is therefore fairly easy to master. Even so, these small commands will become mighty tools in the hands of a skilled Linux administrator later on.