A brief introduction to the Linux Operation System · A brief introduction to the Linux Operation System Xun Chen（谌勋） 2019/7/21 2nd LQCD Training Camp

A brief introduction to the Linux Operation System

Xun Chen（谌勋）

2019/7/21

2nd LQCD Training Camp

Outline

• History of Linux

• Shell operation & Frequently used commands

• Redirection and Pipe

• Remote access

• Text Editors

• Scripting

• Version management

• Window multiplexer

2019/7/21 2nd LQCD Training Camp 2

History


The Creation of Linux

• UNIX was born in the Bell Lab in 1969.• Written in C• For large system and mini-computer (PC is micro-computer)• System-V and BSD

• In 1983, Richard Stallman started the GNU project to create a free Unix-like operating system.• GNU’s Not Unix• GNU General Public License (GPL), “copyleft”• Many software have been created till 1990s.• But a kernel is required

• In 1991, Linus Torvalds created Linux and released it under GPL

2nd LQCD Training Camp 42019/7/21

Events

• 1991, Linus announced Linux kernel publicly.

• 1992, first Linux distribution was created.

• 1993, Slackware and Debian project established.

• 1994, Linux 1.0 released, GUI by XFree86. Redhat and SUSE distributions.

• 1996, Linux 2.0 released, support for several processors.

• 2004, Ubuntu 1.0 released

• 2008, Andorid 1.0 released based on Linux 2.6.25

• 2011, Linux 3.0

• 2015, Linux 4.0


Distributions

• Linux kernel + Software + package management system

• Main distributions• Debian-based: Debian, Ubuntu, Mint, Knoppix

• Gentoo-based: Gentoo

• Pacman-based: Arch

• RPM-based: RHEL, Fedora, CentOS, SUSE

• Slackware-based: Slackware

• Other: Alpine Linux, DD-WRT



Shell operation


Work with bash

• Bash is the default Shell for most of the distributions.

• Type a command and possible options and arguments

• Use the “Tab” key for auto completion, it’s awesome!

• Short-cuts:• Ctrl + a: move cursor to the start

• Ctrl + e: move cursor to the end

• Ctrl + c: End current running program

• Ctrl + d: logout from current session or send “EOF”

• Ctrl + z: suspend a program


ls

• “ls” is the most typed command in Linux

2nd LQCD Training Camp 10

LS(1) User Commands LS(1)

NAMEls - list directory contents

SYNOPSISls [OPTION]... [FILE]...

DESCRIPTIONList information about the FILEs (the current directory by default).

Sort entries alphabetically ifnone of -cftuvSUX nor --sort is specified.

ls -a, ls -l, ls -al

2019/7/21


Try: “ls -l /”, what happened?

“/” is the root directory of all unix-like operation system.

2019/7/21

File and File system

• File: A computer file is a resource for storing information, which is available to a computer program and is usually based on some kind of durable storage.

• File system: In computing, a file system (or filesystem) is used to control how data is stored and retrieved.• space management

• file name

• directory

• types: ext2/3/4, XFS, btrfs… FAT, NTFS…HFS

122nd LQCD Training Camp2019/7/21

Filesystem Hierarchy Standard

• The Filesystem Hierarchy Standard (FHS) defines the directory structure and directory contents in Unix and Unix-like operating systems. • /: root directory of the entire file system • /bin: essential command binary

• /boot: boot loader files

• /dev: devices• /etc: system-wide configuration files

• /home: users’ home directories

• /lib: essential library• /sbin: essential system binaries

• /usr: Secondary hierarchy for read-only user data

• /var: variable files• /tmp: temporary files

• /media: mount point for removable media (CDROM/USB drive)

• /proc: virtual file system, providing process and kernel information


It’s important to know where you are.If you don’t know it, try “pwd”.You can use “cd” to change the current directory.

2019/7/21

File Owner and Permissions

• output of “ls -l”

• file permissions are set for owner, group, others

• read - r, write - w, execute - x

• command “chmod” can used to change the permission of a file• chmod +x file(s)• chmod 644 file(s) (4=r, 2=w, 1=x, 6=4+2)

14

-rw-r--r-- 1 vuser vuser 25454650 Feb 4 2015 geant4.9.6.p04.tar.gz

file permissionfile owner

file group


Do you have the right to access a file?

2019/7/21

PATH - Where are the commands?

• The commands might be:• Built-in commands of bash

• Third-party commands• How do bash know where they are?

• The environment variable tells the CURRENT shell where to look for executables.

• Default $PATH always contains: “/usr/local/bin:/usr/bin:/bin”

• You can use “export” to expand it.export PATH=dir_to_my_favorite_commands:$PATH


Work with Files (I)

• create directory

• mkdir

• remove (delete) files and directories

• rm

• copy files

• cp

• move/rename files

• mv


mkdir my_test_dircd test_dirls -a

touch file.txtrm file.txt

touch file.txtcp file.txt file2.txtls

mv file.txt file3.txtls

2019/7/21

Work with files (II)

• file type:• file

• view text:• cat• more/less

• search for file• find

• search for text in files• grep

• count text file length/lines• wc

• file size• du


file /etc/resolve.conffile /bin/lscat /etc/passwdmore /etc/passwdless /etc/passwd

find /usr/lib/ -name “*ssh*”

grep vuser /etc/passwd

wc -l /etc/passwd

du -s /etc/passwd

2019/7/21

Packaging with “tar”

• You can create packages with the “tar” command

• tar -zcf package.tar.gz file0 file1 …

• The packages can be extracted with

• tar -zxf package.tar.gz


Redirection & Pipes


Default I/O Streams in Linux

• Standard input (stdin, 0)• Usually receive input from a device (keyboard)

• Terminated by EOF (end-of-file), Ctrl+d

• Standard output (stdout, 1)• Write data to the terminal by default

• Standard error (stderr, 2)

• Write error message, default to the terminal


Stream redirection

• Overwrite with single bracket• > - standard output

• < - standard input

• 2> - standard error

• Append with double bracket• >> - standard output

• << - standard input

• 2>> - standard error


Pipes

• Redirect a stream from one program to another• Using the vertical bar |

• The output of the first program is attached with the input of the second program

• Redirections can be chained


Ordinary usage of redirection and pipes

• Command > file

• Command > /dev/null

• Command 2> file

• Command > file1 2>file2

• Command > file 2>&1

• Command >> file

• Command | Command

• Command | tee file

• Command | command | command >> file

• Command < file_input > file_output


Remote AccessIt’s usual to login remote servers.


SSH - Secure Shell

• The most common way of safely administering remote servers• cryptographically secured connection between two

parties

• authenticating each side to the other

• passing commands and output back and forth

ssh -Y username@remote_host

Enable X11 forwarding


Authentication with SSH keys

• No need to supply password

• Use ssh-keygen to generate a pair of keys.• By default a RSA key pair is generated in ~/.ssh/, with

name id_rsa and id_rsa.pub

• Copy the id to the host with ssh-copy-id

ssh-copy-id username@remote_host


Copying files between local machine and remote server• Solution 1: the scp command

• scp local_file username@remote_host:~/destination_dir

• Copy local file(s) to remote host

• scp username@remote_host:~/remote_file local_dir/

• Copy remote file(s) to local directory.

• Solution 2: the rsync command (recommend)• rsync -vaz local_file user@remote_host:~/destination_dir

• Useful options:• -v: verbose

• -q: quiet

• -a: archive mode

• -z: compress file during transfer


EditorsSource code, results files....


Text Editor

• emacs

• vim


How can I get out from the DAMN software?2019/7/21

Editor’s War


Vim Quick Reference


• Two basic mode• Normal mode

• All input are commands

• :q exit

• :w write file

• :wq save and exit

• :x save and exit

• Insert mode• ‘o’, ‘a’, ‘i’ to get into insert mode

• Esc to go back to normal mode

2019/7/21

Emacs Quick Reference

• C for Ctrl, M for Meta (Alt in PC)

• C + x• C + c: exit

• C + f: open (new) file

• C + s: save file

• C + w: save as file

• k: close current buffer (file)


ScriptingYou can group programs and commands together with simple loop control.


Bash programming

• Script starts with magic number: #!/bin/bash

• Commands in text file separated by new line or ;

• Comments start with #

• Terminated when built-in commands fails, continue to next one when non-built-in commands fails


Positional parameters

• Pass parameters to script from command line

• Visit the parameters inside a script using the positional parameters:• $0, $1, $2, ...

• $0 is the command name


Flow control

• Condition

• Loop

if [ -e file ]; thenecho file exists

elseecho file does not exist

fi

for a in {1..10}; doecho $a

done

A complete tutorial in http://ryanstutorials.net/bash-scripting-tutorial/


http://ryanstutorials.net/bash-scripting-tutorial/

Version Control with git


Why version control?

• Keep track of your works, especially for the text-based works

• Bad habit: make copy of the files with date• You may have hundreds copies of a given file

• Hard to manage and search historical data

• With a version control system• Store versions

• Restore previous versions

• Understand the changes

• Backup files


GIT as a local version control system

• GIT is a distributed version control system• Each node has the full history

• You can use it locally

• Three states• Committed

• Modified

• Staged


Useful commands

• Initialize a repository• git init

• Add modified file(s) to stage area• git add file_name

• Commit the staged changes• git commit -m “the reason of updates”

• Read the commit history of current branch• git log

• Check current status of the working tree• git status

• Create new branch based on current commit• git branch branch_name

• Checkout a given branch or commit• git checkout branch_name | commit number

• Merge a branch to current branch• git merge branch_name


Window multiplexerTo avoid the annoying network problem


tmux

• Multiple terminal sessions in the same window


Using tmux

• Start tmux session• tmux• tmux new -s session_title

• Detach from a tmux session• Ctrl-b d

• Attach to an existing session• tmux a• tmux a -t session_title

• Splitting pane• Ctrl-b ‘• Ctrl-b %


Documents

A brief introduction to the Linux Operation System · A brief introduction to the Linux Operation System Xun Chen（谌勋） 2019/7/21 2nd LQCD Training Camp