User Tools

Site Tools


files-and-directories

Files & Directories


FILES & DIRECTORIES
/ represents a root directory
~ / ~<user> represents a home directory of the logged in / particular user
. represents a current (working) directory
.. represents a parent directory
pwd prints the name of the current (working) directory
cd / cd ~ / cd $HOME
cd <path_to_directory>
changes the current directory to a user's home directory
changes the current directory to a particular directory; the path is either absolute (beginning with "/") or relative (specified by the directory name - if the given directory is under the current directory, or by ".." or "-", which represents a parent or previous directory)
tree / tree <directory> lists contents of the current / particular directory in a tree-like format, -a including hidden files, -d directories only, -L <n> up to the given maximum directory tree level
$ tree -a | tail -1
(lists a total number of directories and files within the current directory)
ls / dir (<file / directory>) prints a file name / contents of a directory alphabetically, -a including hidden files, -d directory itself, without its contents, -i i-node number, -l detailed output (permissions, number of hard links, owner, group, size in B, time of last change, file / directory name), -r reverse order, -t sorted by last change, -u sorted by last access, -A does not list implied "." and "..", -F indicates an item type ("*" = executable file, "@" = symbolic link, "/" = directory, "=" = socket, "|" = named pipe), -R recursively, -S sorted by size, -X sorted by entry extension, -Z prints a security context within SElinux
$ ls -ld data
drwxrwxr-x+ 2 root root 4096 Oct 4 17:39 data
(the output says that there are ACL permissions set up on the directory (character "+” at the end of the permissions) and it is writable by the group, in this case, however, the output of "getfacl” command is relevant, which usually shows that the group has no write access to this directory, but it means the directory is writable by a user with ACL privileges)
$ ls -l /var/lib/nfs/etab
-rw-r--r--. 1 root root 0 Apr 16 2010 /var/lib/nfs/etab
(the output says that there are special attributes set up on the file (character ".” at the end of the permissions; these attributes can be shown by "lsattr" command)
file <file> determines file data type (ASCII, PDF, HTML...)
$ file * | grep ASCII
stat <file> prints detailed file attributes (name, size in B, number of blocks, file type, i-node number, permissinos, owner and group, time of last access, modification and change), -c <format> uses the specified output format (%a octal permissions, %A symbolic permissions, %b number of blocks, %F file type, %g group ID, %G group name, %i i-node number, %n file name, %s size in B, %u owner ID, %U owner name, %x time of last access, %y time of last modification, %z time of last change), -f information about the FS
$ stat -c "%U %G" file
du (<file / directory>) displays a disk usage size of the particular file / directory with its contents in kB; without a parameter the size of directories within the working directory is displayed only, -a displays the size of all files and directories recursively, -m displays the size in MB, -h well-arranged output, -s total summary without details
# du -sh /home/*
# du -am /var/log | sort -rn | head -50
(displays 50 biggest files and directories within the particular directory)
touch <file> creates an empty file, if it exists, it changes last access and modification time to the current time, -a changes last access time only, -m changes last modification time only, -d "<YYYY-MM-DD hh:mm>" sets the given time instead of the current one, -c does not create any file
mkdir <directory> creates a directory, -m <permissions> sets access permissions, -p creates several child directories at the same time (if they do not exist already)
$ mkdir -p /dir1/dir2/dir3
rm <file / directory> removes a file, -R directory with its contents, -i asks for confirmation, -f without confirmation, -v detailed output
rmdir <directory> removes an empty directory, -p more child directories at the same time
cp <source> <target> copies a source file into an existing target directory; if two files are considered and the target one does not exist, it will be created, if it exists, its contents will be automatically overwritten, -i asks for confirmation, -f without confirmation, -b creates a backup of the target files that might be eventually overwritten, -R copies a diretory with its contents, -p preserves the source attributes, -u copies only when the source is newer than the target or the target is missing, -v detailed output
# cp -pR /home /mnt/extdisk
# cp -pR /home/tom/{*,.[^.]*} /mnt/extdisk/home/tom
(copies the contents of user "tom” home directory to an external disk, including the hidden files)
rsync <source> <target> copies a source file into an existing target directory; if two files are considered and the target one does not exist, it will be created, if it exists, its contents will be automatically overwritten; if the target already contains some identical data as the source, only the difference is being copied; it is also possible to tranfer data between remote computers, -r copies a diretory with its contents, -l preserves symbolic links, -p preserves permissions of the file, -o preserves the owner of the file, -g preserves the group of the file, -t preserves modification times, -D preserves special files, -a includes options "-rlptgoD", -z compresses the data during the transfer, --delete removes redundant data in the target directory which do not exist in the source any more, --ignore-errors removes redundant data even in case of I/O errors, --exclude <pattern> excludes files matching the given pattern, --progress prints the progress of synchronization, --rsh <command> specifies a remote command during the transfer, -v detailed output
# rsync -a --delete --exclude ".gvfs" /home /mnt/extdisk
$ rsync --rsh="ssh -l root -p 22022" file1 file2 192.168.152.47:/tmp
dd <parameter>=<value> copies data between devices, if=<source>, of=<target>, bs=<block_size_in_bytes> (512 B by default), count=<number_of_blocks>
$ dd if=/dev/cdrom bs=1500 of=/tmp/image.iso
$ dd if=/dev/zero of=test bs=5M count=10
(easily creates an empty bigger file)
# dd if=/dev/urandom of=/dev/sdb1 bs=1G
(overwrites a disk / partition with random data)
mv <source> <target> moves a source file or directory into an existing target directory; if two files or two directories are considered and the target one does not exist, the source one will get the target`s name, if the target file exists, its contents will be automatically overwritten, -i asks for confirmation, -f without confirmation, -b creates a backup of the target files that might be eventually overwritten, -v detailed output
rename <source> <target> <file / directory> renames a particular file or directory from the source name to the target name according to the given patterns
$ rename .htm .html *.htm
$ find / -name "honza*" | xargs rename honza jan
ln <source> <target> creates a hard link to a file / directory, -s symbolic link
find <path> <option> <string> (<action>) searches for a file / directory, -name <name> according to the name, -iname <name> according to the name (the match is case insensitive), -regex <pattern> according to the name which matches a particular regular expression, -iregex <pattern> according to the name which matches a particular regular expression (the match is case insensitive), -inum <n> according to the i-node, -atime / -amin <-n / n / +n> according to the last access in items of days / minutes, -ctime / -cmin <-n / n / +n> according to the attributes change in items of days / minutes, -mtime / -mmin <-n / n / +n> according to the contents change in items of days / minutes, -size <-n / n / +n><size> according to the size ("k” = kB, "M” = MB, "G” = GB), -type according to the type ("d” = directory, "f” = file, "l” = symbolic link, "b” = block device, "c” = character device), -context <context> according to the SELinux security context, -user <user / UID> according to the user, -group <group / GID> according to the group, -perm <permissions> exactly according to the given permissions, -perm -<permissions> according to the set permissions whose all bits includes at least the given permissions bits, -perm /<permissions> according to the set permissions whose any bit also includes the given permissions bit, -nouser searches for files of nonexisting users, -nogroup searches for files of nonexisting groups, -exec <command> (<arguments>) {} \; executes a command (string "{}" is replaced by the found file`s path, ";" represents the end of a command, protected from the shell expansion), -ok <command> (<arguments>) {} \; interactive version of "-exec" option (asks for confirmation), -maxdepth <level> searches up to the given maximum directory tree level, -mindepth <level> searches minimally from the given directory tree level, -print prints the result, -print0 prints also file names containing a white space, -o = "or"
'# find /home -name "*.jpg" -o -name "*.png"
# find . -type f -o -type d | wc -l
(counts all files and directories within the working directory)
# find ! -type f
(searches for everything except for ordinary files within the working directory)
# find / -user nike -exec chown tom {} \;
# find / -user tom -exec rm -Rf {} \;
# find . -maxdepth 2 -type l
# find / -nouser -o -nogroup
# find / -perm -4000
(searches for all files with SUID bits set)
# find / -context "*tmp_t*"
# find /var/log -type f -size +100M | xargs du -am | sort -rn | head -10
(searches for 10 files bigger than 100 MB and sorts them by size in descending order )
# find /etc/rc.d -type f \( -perm -g=w -o -perm -o=w \)
(searches for writable files for group or others excluding symbolic links)
# find / -type f -regex ".*\.\(sh\|pl\|py\)" -perm -o=x
(searches for executable scripts for others)
which <program> displays a path to a binary file of the particular program (its alias), -a prints all results in $PATH
whereis <program> displays a path to a binary, source and manual file of the particular program


The Contents of a File
od (<file>) (-o) prints the contents of a binary file in octal format, -d in decimal format, -x in hexadecimal format, -j <byte> prints data from the particular byte (offset), -w<number_of_bytes> displays the given number of bytes on one line; the first column of the output represents an offset - the sequence of the first byte of the particular line from the beginning of the file; if no file is specified, reads from STDIN
$ od -w8 /usr/bin/who
$ od -j 0474360 /usr/bin/find
strings <file> prints text strings contained especially inside a binary file with a minimum length of four characters
# strings $(which vsftpd) | egrep "libwrap|hosts"
(checks if the program is related to tcp wrappers)
cat (<file>) prints the contents of a text file, -n numbers all lines, -b numbers lines containing a text; if no file is specified, reads from STDIN
$ cat a b c > abc
cat > <file> inserts the contents of STDIN into a file
$ cat > /dev/pts/2 <-'
<text> <-'
ctrl+d
tac (<file>) prints the contents of a text file in reverse; if no file is specified, reads from STDIN
more <file> prints the contents of a text file one screenful at a time
less <file> prints the contents of a text file one screenful at a time (keys "page up/down" available)
head (<-n>) (<file>) prints first n lines of a file (10 by default); if no file is specified, reads from STDIN
tail (<-n>) (<file>) prints last n lines of a file (10 by default), -f refreshes the output as the file grows; if no file is specified, reads from STDIN
$ head -25 access.log | tail -1
(prints 25th line of the file)
wc (<file>) prints the number of lines, words and bytes in a file including white spaces, -l counts lines, -w words, -m characters, -c bytes; if no file is specified, reads from STDIN
nl (<file>) numbers lines of a file, if no file is specified, reads STDIN, -v <n> sets a number that starts the numbering (by default 1), -i <n> sets an increment by which the number of the following line is increased (by default 1), -b <option> numbers lines corresponding to the given option (a all lines, t nonempty lines (by default), n no lines, p<expression> lines containing a particular regular expression), -s <string> sets a separator between the number of the line and its contents (by default tabulator), -w <n> sets the number of characters to align the particular figures to the right (by default 6); if no file is specified, reads from STDIN
$ nl style.css | more
$ nl -s ": " -b p^# script.sh
grep <expression> (<file / directory>) searches a text file for strings containing a particular expression (the whole line is being printed), -v prints the lines not containing the given expression, -n prints the line number, -c prints the number of lines containing the particular expression only, -l prints the names of files containing the particular expression only, -L prints the names of files not containing the particular expression only, -i ignores case distinctions, -f <file> reads expressions from a given file (one per line), -o prints only the part of a line matching the expression, -E interprets the expression as an extended regular expression (see "egrep" command), -R recursively, --color displays the matched string in color; if no file is specified, reads from STDIN
$ grep linux notes / cat notes | grep linux
$ grep '^P.*r$' notes
$ grep -E 'AB | CD' notes
$ grep -R linux ~/data
$ ls -l | grep '^d'
egrep <expression> (<file / directory>) interprets the expression as an extended regular expression (see "grep -E"); if no file is specified, reads from STDIN
$ egrep -o '^[^:]+' /etc/passwd
(displays all users in the system)
uniq (<file>) prints the contents of a text file discarding all but one of successive identical lines, -d prints duplicate, successive lines only (to print all duplicate lines the contents of the file has to be sorted), -u unique lines, -c prefixes lines by the number of occurrences, -i ignores case distinctions; if no file is specified, reads from STDIN
sort (<file>) sorts lines of a text file in ascending order, -n in numerical order, -f ignores case distinctions, -u ignores duplicate expressions, -r in reverse order; if no file is specified, reads from STDIN
$ sort file.txt | uniq
tr <'old_character'> <'new_character'> replaces single characters from the first string by the new characters from the second one, the original file is not being overwritten, -d removes characters
$ cat file1 | tr 'a-z' 'A-Z' > file2
$ cat text1 | tr ' ' '\t' > text2
(each space is replaced by a tab in the file)
$ tr -d ';' < text1.txt > text2.txt
sed (<address>) <command> (<file>) filters and edits a text in a noninteractive way (often used in scripts), -e <script> executes a particular script, -f <file> executes a script included in the given file, -i saves the changes directly in the file (otherwise changes need to be saved by using redirections), -n does not print the output on STDOUT; the address represents either a sequential number from the beginning of the input (e.g. 1,10 = range of lines), "$" character (last line of the input) or a pattern (regular expression) bounded from the both sides by a separator - normally by "/"; command s replaces the pattern by the following expression bounded by separators (only the first matching pattern), n exactly n-th pattern occurence, g replaces the pattern globally, in each occurence, <n>p prints a particular line of the file (with option "-n"), d removes the whole line containing the particular expression, !<command> executes a command for all addresses except for those specified; if no file is specified, reads from STDIN
$ echo "11 x, 22 x, 33 x" | sed 's/x/y/2'
(replaces only the second occurence of "x" pattern by "y")
$ sed -i 's/string1/string2/g' file
(replaces the first string by the second one within the whole file)
$ sed 's/ \+/\t/g' test1 > test2
(replaces multiple spaces by one tab)
$ echo "Log retention:" $(sed -n '3p;6p' /etc/logrotate.conf)
(prints 3rd a 6th line of the file)
$ sed '/dev/!d' file
(prints all lines containing a "dev" expression)
$ sed -i '1,10d' file
(removes first 10 lines of the file)
$ sed -i '5,$d' file
(removes all from the 5th line to the end of the file)
$ sed '1,/START/d' file
(removes all up to and including the word "START")
$ sed '1,/STOP/!d' file
(removes all from the word "STOP")
$ sed -e '/^#/d; /^$/d' file1 > file2
(removes all comments and empty lines in the file)
$ sed '/^[^#]*\(sha[25]\|md5\)/!d' /etc/pam.d/system-auth /etc/pam.d/common-password 2> /dev/null | egrep -o "sha[1256]{3}|md5" || sed '/^[^#]*ENCRYPT_METHOD/!d' /etc/login.defs
(displays encryption algorithm for newly created passwords)
awk '<program>' (<file>) filters a text in a noninteractive way (often used in scripts); the program is a set of rules containing a pattern, action, or both, the action is separated by "{}"; -f <file> reads a program from the given file, -F <string> sets input items` separator, -v <variable>=<value> sets a particular variable within the program; the following built-in variables are available NR number of records (lines) separated by a new line, NF number of fields in the record separated by a separator (white space by default), particular fields are represented by $1, $2 etc., $0 represents the whole record, RS defines records separator, FS defines fields separator, ORS defines output records separator, OFS defines output fields separator; the following operators are used to compare strings == is equal to, != is not equal to, < less than, > more than, <= less than or equal to, >= greater than or equal to; regular expressions are delimited by "/""/", for comparison use ~ pattern matches RE, !~ pattern does not match RE; || logical "or", && logical "and"; if no file is specified, reads from STDIN
$ awk '/nameserver/ {print}' /etc/resolv.conf
(prints all lines containing "nameserver")
$ awk '{print $1, $2}' /etc/fstab
(prints each first and second word of each line of the file)
$ awk -F ":" '{print $1, $7}' /etc/passwd
(prints all users and their login shells)
$ awk '{print $1 > "file1"; print $2 > "file2"}' test
(copies the first and second word of the file to the given files)
$ awk 'length > 10' list.txt
(prints lines longer than 10 characters)
$ awk 'NF < 5' list.txt
(prints lines shorter than 5 words)
$ awk 'NR == 5' error.log
(prints 5th line of the file)
$ awk 'NR == 5, NR == 10' album
(prints 5th up to 10th line of the file)
$ awk '{print NR,"->",NF}' list.txt
(prints number of fields for each record of the file)
# ne=$(awk -F ":" '{if (length($2) > 2 && ($5 == "" || $5 >= 99999)) print $1}' /etc/shadow); [[ -z "$ne" ]] && echo "NONE" || echo "$ne" | tr " " "\n"
(prints users with non-expiring passwords)
vi / vim <file>
ESC
:<n>
:q
:q!
:w / :w <file>
:wq / :x / ZZ
:set number / nonumber
:set ignorecase / noignorecase
:set all

moving the cursor:
left:
h / <left arrow>
b
0
right:
l / <right arrow>
w
$
up:
k / <up arrow>
H
1G
down:
j / <down arrow>
L
G

adding a text:
i
a
I
A
o
O

removing / extracting a text:
x
db
dw
d0
d$
dd / <n>dd
d1G
dG

modifying a text:
yy / <n>yy
P / p
r
R
:(<address>)s/<searched_string>/<replacing_string>(/<options>)




u
U
.

searching in a text:
/<expression>
?<expression>
n / N
starts a "vi" text editor and creates or opens a file
gets into the command mode
moves the cursor at the beginning of a particular line
leaves the editor (provided the file was not changed)
leaves the editor without saving eventual changes in the file
saves the changes in the file / saves the file as a new one
saves the changes and leaves the editor
numbers the lines / removes line numbering
ignores / respects case distinctions
prints option settings



one character to the left
at the beginning of the previous word
at the beginning of the line

one character to the right
at the beginning of the following word
at the end of the line

on the previous line
on the first line of the screen
on the first line of the file

on the following line
on the last line of the screen
on the last line of the file


at the current cursor position
after the current cursor position
at the beginning of the line
at the end of the line
at the beginning of a new line bellow the current line
at the beginning of a new line above the current line


one character in the current cursor position
characters before the current cursor position to the beginning of the word
characters from the current cursor position to the end of the word
characters before the current cursor position to the beginning of the line
characters from the current cursor position to the end of the line
the whole line in the current cursor position / next n lines
the whole line in the current cursor position to the beginning of the file
the whole line in the current cursor position to the end of the file


copies the current line / following n lines
pastes the text in buffer on the current / following line
replaces a character in the current cursor position
replaces characters in the current cursor positions
replaces the searched expression with a new one; address 1,$ specifies the range of the whole document, if the address is not given, the change affects the current line only, option g replaces all appearances of the searched string, option c asks for confirmation to procede
:1,$s/hundred/100/g
(replaces all appearances of "hundred" with "100" within the whole file)
restores the last performed change
restores all changes made on the current line
repeats the last command


searches the file downwards for the specified string
searches the file upwards for the specified string
continues in further searching / searching in the reverse direction
paste <file1> <file2> joins contents of files one next to the other (1 column = 1 file), -d <string> defines a field separator instead of a tab, -s transposition of lines and columns
$ paste -s file1 file2 > file3
cut (<file>) -c <m-n> cuts a given range of characters on each line of the file (meant from the beginning of the line), -f <m-n> cuts a given range of columns (fields) separated by a tab, -d <string> defines a field separator instead of a tab; if no file is specified, reads from STDIN
$ cut -f 2,4 -d ' ' data
split (<file>) splits a file into pieces of the same number of lines (1000 by default), -l <n> specifies the number of lines, -b <n> splits a file by the given number of bytes; if no file is specified, reads from STDIN
join <file1> <file2> joins lines of two sorted text files according to the first identical field usually delimited by whitespace, -i ignores case distinctions
diff <file1 / dir1> <file2 / dir2> compares contents of two text files or directories and prints differences, -i ignores case distinctions, -q reports only whether the files differ, -r recursively, -u prints the output in a unified format
cmp <file1> <file2> compares contents of two files of any type and prints the first different byte and line number, -l prints all differences byte by byte
cksum / sum (<file>) prints CRC checksum of a file and its size in B / kB (used to check the identity of files; different algorithm); if no file is specified, reads from STDIN
sha1sum (<file>) prints or checks SHA1 (160-bit) checksums of a file, -c <file> checks SHA1 sums against the given value contained in the file; if no file is specified, reads from STDIN
$ sha1sum /home/user/F-7-x86_64-DVD.iso
$ sha1sum page.txt > page.sha1
$ sha1sum -c page.sha1
md5sum (<file>) prints or checks MD5 (128-bit) checksums of a file, -c <file> checks MD5 sums against the given value contained in the file; if no file is specified, reads from STDIN
$ md5sum /home/user/F-7-x86_64-DVD.iso
$ md5sum page.txt > page.md5
$ md5sum -c page.md5
> <file> / cat /dev/null > <file> removes the contents of a file
enca <file> prints the type of coding of a text file
$ ls > list; enca list
convmv <file / directory> changes the type of coding of a file / directory name, -f <source_code> defines the original code, -t <target_code> defines a new code, -i interactive mode, -r recursively, --notest ensures real recoding of a file name
$ convmv --notest -f cp1250 -t UTF-8 *
recode <source_code>..<target_code> <file> changes the type of coding of a text file, -l prints types of coding, -f suppresses error output, -v detailed output
$ echo 'List of all file names' | recode -f UTF-8..flat
$ recode ..HTML < page.txt > page.html
$ find . -name "*.txt" -exec recode cp1250..UTF-8 '{}' \;
convert <source> <target> changes picture file formats and eventually the file sizes
$ convert image.tif image.jpg
aplay <file> plays a particular audio file
arecord <file>.wav creates a "wav" audio file
lame <file>.wav <file>.mp3 changes audio format from .wav to .mp3 or vice versa, -b <bitrate> defines min. bitrate, -v variable bitrate, --verbose detailed output


Archiving & Compression
tar <target> <source> archives or restores data (the source remains), -c creates an archive, -f defines a file name, -v detailed output, --remove-files removes the source, -M multi-volume archive (data requires more than one medium), -V <name> defines a volume name, -X <file> excludes files listed in the given file, -t prints the contents of an archive, -x extracts files from an archive, -p extracts data with the original ownership and permissions, -d compares data in an archive with data on the disk, -z compress data by gzip, -j compress data by bzip2, -w interactive mode
# tar -cvf archive.tar /home/user/xxx
# tar -cMf /dev/st0 / -V system_bckp
# tar -xvf archive.tar
# tar -tvf archive.tar
for archiving and (de)compression by gzip:
# tar -cvzf file.tar.gz /home/user/xxx
# tar -xvzf file.tar.gz
for archiving and (de)compression by bzip2:
# tar -cvjf file.tar.bz2 /home/user/xxx
# tar -xvjf file.tar.bz2
# tar --remove-files -cvjf maillog_old.tar.bz2 maillog-20161225 maillog-20170102 maillog-20170108
(<source> |) cpio > <target> archives or restores data, -o creates an archive, -d creates a directory tree, -v detailed output, -i extracts an archive, -m retains previous file modification time, -t prints the contents of an archive
$ ls | cpio -ov > archive.cpio
# ls /home | cpio -ov > /dev/rmt0
# find / -mtime -1 -type f -print | cpio -ov > /dev/rmt0
$ cpio -o > /dev/fd0 <-'
<file1> <-'
<file2> <-'
ctrl+d
# cpio -ivmd /home/\* < /dev/st0
gzip <file> compresses a file by gzip (a directory must be first archived into one file by tar; the source is removed), -<n> compression level (the value range is from 1 - 9, the higher value the higher compression, default value is 6), -c prints the output on STDOUT and retains the source, -v detailed output
# gzip -c access.log > access.log_2017-02-15.gz && > access.log
(compresses a file which is saved under a new name and removes the contents of the original file)
gunzip / gzip -d <archive> decompresses a file, -c prints the output on STDOUT and retains the source, -l prints an archive name, its size in B before and after the compression and the compression ratio, -t tests an archive contents, -v detailed output
$ gzip -cd file.tar.gz | tar -x(f -)
bzip2 <file> compresses a file by bzip2 (higher compression; a directory must be first archived into one file by tar; the source is removed), -<n> compression level (the value range is from 1 - 9, the higher value the higher compression, default value is 6), -v detailed output
bunzip2 / bzip2 -d <archive> decompresses a file, -c prints the output on STDOUT, -k retains the source, -t tests an archive contents, -v detailed output
$ bzip2 -cd file.tar.bz2 | tar -x(f -)
zip <target> <source> compresses a file by zip, -r recursively, -e under a password
$ zip -r data.zip data
unzip <archive> decompresses a file, -l lists an archive contents, -t tests an archive contents
unrar <archive> e extracts an archive to the current directory, l lists an archive contents, t tests an archive contents
Last modified: 2017/04/09 20:08 by Miroslav Bernát

visits: