Follow by Email

Tuesday, August 16, 2011

Usage Of "lsof" Command

The lsof is an excellent utility that can be used to identify process or user that is locking system resource such as file or network socket.

Basically, The /etc/services system files is usually used to map a numeric port number to a descriptive port name defined by user.


lsof without any command options to list system wide resources that are using by processes running in the system.

To report all processes that are accessing the TCP sockets found on the system
lsof -i TCP

To find out what process is holding TCP port 8080.
lsof -i tcp:8080

To show all resources that are being held by process id 857
lsof -p 857

Some programs might running on the Linux servers by more than one instance. 

In this case, type lsof -c ProgramName instead of lsof -p PID to get a broader scope of view.

For example:

To find out what are the resources held by all sendmail processes running on the system.

lsof -c sendmail

To confirm resources that are being held by user id user1
lsof -u user1

To find out what are the processes that are locking the specify file /home/user1/myfile
lsof /home/user1/myfile

Friday, August 12, 2011

Backup process in AIX

7.1 AIX
backup and restore are proprietary AIX commands that provide file system dumps and recovery as well as backup of individual files and directories. These commands resemble dump and restore and are provided in addition to the standard UNIX utilities of tarcpiodd, and pax. Doing a file system dump with the backup command is known as "backing up by i-node format" in AIX parlance. The basic syntax of backing up a file system by i-node is:
# backup -f Device -DumpLevel -u File system
Device can be a file or a device file. DumpLevel is a numeric value from 0 to 9, where 0 represents a full file system backup. The other dump levels represent incremental backups: an n level backup includes all files modified since the last n - 1 level backup. If you do not specify a dump level, backup defaults to a level 9 backup. The -u option updates the /etc/dumpdates file, a log of your backups used by backup when doing an incremental backup. You should unmount a file system before backing it up, the root file system being the exception, of course. If you dump the root file system, file systems mounted on root, including journaled file systems, are not backed up. [6]
Do a full backup of the /home file system to tape, rewinding upon close:
umount /home
# backup -f /dev/rmt0 -0 -u /home
mount /home
Back up the root file system's files that have changed since the last 0-level backup to tape, rewinding upon close:
# backup -f /dev/rmt0 -1 -u /
In the first example we used the mount point of /home as the name of the file system. You can do that; in fact, it is easier to remember to do it this way. However, the /etc/dumpdates file will record the name of the file system using the raw device file instead of the mount point. In the case of /home that raw device file is /dev/rhd1 (by default the /home file system's device files are /dev/hd1 and /dev/rhd1). Don't be confused by this. If you prefer you can use either the raw device or block device file name in the backup command instead of the mount point. To get the block device file name of a file system, use the lsfs command. From that you can determine the raw device file name by prepending the letter r to the block device file name.
To back up individual files and directories use the -i option (for individual files). This is also known as backing up by name. The basic syntax is:
# backup -f Device -i
To use this option you must provide backup with a list of names as standard input by typing the names of each file after typing the command (ending the list with a Ctrl-D), using redirection, or using either the find command or the cat command. In the examples that follow the -q option is used to prevent AIX from prompting you to mount the backup media:
# backup -f /dev/rmt0 -iq
# backup -f /dev/rmt0 < /tmp/filelist -iq
# cat /tmp/filelist | backup -f /dev/rmt0 -iq
# find /home -print | backup -f /dev/rmt0 -iq
The last example above will back up the contents of the entire /home directory, but in the two examples before that, if you have /home listed in /tmp/filelist, only the directory name will be backed up. To construct the command in KornShell syntax so that it backs up directory trees as well as files, type the following:
# find $(< /tmp/filelist) -print | backup -f /dev/rmt0 -iq
To see a list of the files and directories scroll on your screen while backup is running, use the -v option. To save this output, redirect standard output or use the tee command.
# find $(< /tmp/filelist) -print | backup -f /dev/rmt0 -iqv >\ /var/adm/backed.up.files
# find $(< /tmp/filelist) -print | backup -f /dev/rmt0 -iqv | tee |\ /var/adm/backed.up.files
The restore command is used to restore files backed up by i-node or by name. This command works only on archives created by the backup command and is capable of determining which format, i-node or by-name, was used for the backup. The basic syntax of restoring entire file systems is:
# restore -f Device -r
The basic syntax of restoring files backed up by name is:
# restore -f Device -x
Restore an entire file system backed up by i-node from tape.:
# restore -f /dev/rmt0 -r
You do not have to unmount the file system before restoring in this manner, but be sure to change directories to the appropriate file system first because restore assumes relative path names.
To restore individual files backed up by i-node, use the -i option. This will start restore in interactive mode, which will step you through the process of restoring your files. For example:
# restore -f /dev/rmt0 -i
Extract all files backed up by name from tape:
# restore -f /dev/rmt0 -x
Extract all files from tape and display file names while doing so (-v, for verbose):
# restore -f /dev/rmt0 -xv
Extract verbosely the file /home/partlist:
# restore -f /dev/rmt0 -xv /home/partlist
Extracts the entire /home directory from tape verbosely:
# restore -f /dev/rmt0 -xv -d /home
To avoid frustration in restoring files, it is extremely important that you know whether files backed up by name were done so using full or relative pathnames. To find out, use the -T option to get a list of files, for example:
List files from the file /archive/backup1 created by the backup command:
# restore -f /archive/backup1 -T
List files from tape :
# restore -f /dev/rmt0 -T
Use the names of the files or directories exactly as listed in the output, even if the path names begin with a "dot", such as ./home/partlist
AIX provides a means to create an bootable tape containing an image of the root volume group. This can be a life saver for single-disk systems in that you have a very quick means of recovery in case of disk failure, provided, of course, your mksysb image is fairly current. You can also use mksysb to install other machines, although this will require redoing some of the configuration, such as IP addresses, because virtually everything on the original machine is copied to tape.
To do a system image backup:
1.     Start SMIT:
2.     You will see the following:

Backup the System
Type or select values in entry fields.
Press Enter AFTER making all desired changes.

                                                        [Entry Fields]
    WARNING:  Execution of the backup command will
               result in the loss of all material
               previously stored on the selected
               output medium. This command backs
               up only rootvg volume group.

FORCE increase of work space if needed              no                 +
* Backup DEVICE or FILE                            []
   (example: /dev/rfd0)

Simply insert the device file name of your tape drive, such as /dev/rmt0, and choose yes for FORCE increase of work space if needed. The latter is necessary if your system takes more than one tape for the backup.
Keep in mind that mksysb creates an image backup of the root volume group only! If you have other volume groups, you must have separate backups for those.
It is important to understand how AIX handles tape drives because you may find yourself unable to restore files from a tape you thought contained a good backup. One of the attributes in AIX's configuration of a tape drive is that of block size, the amount of data read or written in a single operation. When a tape drive is added to AIX, a default block size is configured for you. Frequently that block size is 512 bytes. However, you can change the block size through SMIT or with the chdev command. If you have a backup tape written with a block size of 512 bytes, but your AIX driver reads in 1024-byte blocks, you will not be able to restore the contents of the tape unless you change the block size to 512 bytes. For example:
# chdev -l rmt0 -a block_size=512
If you want, you can let the application or command, such as dd, determine the block size, in which case you can specify that your tape drive block size is variable in length. To do so use 0 as the block size:
# chdev -l rmt0 -a block_size=0
If you are reasonably confident that you have a valid backup on tape, and you just can't read it no matter what you do, try changing the block size of the tape device until it works. Just remember: restoration of data from a tape can only work if the block size used in the restore is the same as that of backup block size.
For tape operations AIX uses the tctl command, similar to HP-UX's mt command. The syntax of tctl is:
# tctl -f Device Subcommand Count
The most common tctl subcommands are:
Moves the tape forward the number of file marks specified by Count.
Moves the tape backwards the number of file marks specified by Count.
Rewinds the tape.
Erases all contents on the tape and rewinds it.
Fast-forwards, then rewinds the tapeBackupsAIXbackup_AIXAIXBackupsbackup_AIX
7.2 HP-UX
Like AIX, HP-UX has a proprietary scheme for backups in addition to the standard UNIX utilities. This scheme includes the fbackup and frecover commands. The basic syntax of the fbackup command is:
# fbackup -f Device [0-9] [-u] [-i path] [-e path] [-g graph]
Because fbackup does not by default write to standard output, the -f option is not optional. Device can be a file, device file, or a remote device file. A remote device file takes the form machine:/dev/device_name. You can specify - as the device to have fbackup write to standard output. The [0-9] option provides for incremental backups using the same scheme as that for AIX's backup: an n level backup includes all files modified since the last n - 1 level backup.
The -i-e, and -g options provide a means of specifying which portions of a file system you want backed up. If you recall, AIX's backup command backs up either entire file systems, or individual files. fbackup allows you to include a file system with the -i option and exclude portions of it with the -e option. For example:
# fbackup -f /dev/rmt/c201d3m 0 -i /usr -e /usr/tmp
This command does a full backup to tape of the /usr file system with the exception of the /usr/tmp directory tree. You can also place your inclusion and exclusion parameters in a graph file. A graph file is a text file containing the path names of the files and directories you either want included or excluded from your backup. These path names are preceded with either an i (denoting inclusion) or an e (denoting exclusion). For example, if your graph file contains the following lines:
i /usr
e /usr/tmp
e /usr/lib
i /users
e /users/guest
Then the /usr file system will be backed up with the exception of the /usr/tmp and /usr/lib directory trees, and the /users file system will be backed up with the exception of the /users/guest directory tree. If the name of this graph file is /usr/local/backup/graph1, then your command would look something like:
# fbackup -f /dev/rmt/c201d3m 0 -g /usr/local/backup/graph1
fbackup uses the usr/adm/fbackupfiles/dates file to record backup information for purposes of doing incremental backups. (This is similar to AIX's /etc/dumpdates file.) This file is not updated unless you use the -uoption, which is only available with the -g option.
Perform a level-8 backup to /dev/rmt/c201d0m using the graph file /usr/local/graph, and update the /usr/adm/fbackupfiles/dates file:
# fbackup -f /dev/rmt/c201d0m 8 -u -g /usr/local/graph
Back up everything under the current directory except subdir to /dev/rmt/c201d0m:
# fbackup -f /dev/rmt/c201d0m -i . -e ./subdir
Back up everything under the current directory except subdir to /dev/rmt/c201d0m on system roberts:
# fbackup -f roberts:/dev/rmt/c201d0m -i . -e ./subdir
The frecover command is the counterpart to the fbackup command. frecover can restore only files backed up with fbackup. There are four basic modes of operation for frecover.
Recover everything on a backup volume:
# frecover -r
Extract certain files from a backup volume:
# frecover -x
Read the index from the backup volume and write it to path:
# frecover -I path
Restart an interrupted recovery:
# frecover -R path
As you can see, fbackup resembles AIX's restore command's options in the form of -r and -x. frecover -r and frecover -x have some options in common:
frecover -x | -r [ -fhoFX ]
frecover defaults to /dev/rmt/0mn, but you can use -f to specify a different device, including a remote device. The -h option is used to restore only directories but not the files contained in them. The -o option is used to force frecover to overwrite a newer file with an older one. Normally frecover does not do this. The -F option causes frecover to strip all the leading directories from the path names of files being recovered. This allows you to restore files backed up with full path names to different directories. The -x option makes all recovered files relative to the current directory. If, for example, you restored the file /usr/bin/vi and your current directory was /users/root, then the restored file's new path would be /users/root/usr/bin/vi.
For the frecover -x mode the -g option is available. This allows you to use a graph file in the same way as with fbackup. The file format of the graph file is the same. This is useful for partial recoveries.
Recover all files from medium-density tape:
# frecover -rf /dev/rmt/c201d0m
Recover all files indicated in /usr/local/graph:
# frecover -x -g /usr/local/graph -f /dev/rmt/c201d0m
Retrieve an index of files from tape and put it in /tmp/index:
# frecover -I /tmp/index -f /dev/rmt/c201d0m
HP-UX and Cartridge Tapes: tcio
The wear and tear on a cartridge tape drive is extensive when redirection is used because the data transfer rates between the host computer and the cartridge tape drive are not synchronized. Thus HP developed the tciocommand to "buffer up" data transfer between the backup command and the cartridge tape drive. Instead of redirecting the output of the backup command straight to the device, it is piped through tcio to enable streaming to occur. tcio can be used with cpio as well as with fbackup.
Make relative backup on cartridge tape /dev/rct/0s0:
# find . -print | cpio -o | tcio -o /dev/rct/0s0
Back up /users to cartridge tape:
# fbackup -f - -i /users | tcio -o /dev/rct/0s0
HP-UX doesn't have the mksysb command nor a command that all by itself does the same thing. For one thing LVM doesn't exist in 9.x for the 700 series. But you can create a bootable image of a disk by taking the following steps:
1.     Shut down to single user mode to minimize system activity:
# shutdown 0
2.     Clear any remaining data in the buffers:
# sync;sync
3.     Use dd to copy the LIF boot area:
# dd if=/usr/lib/uxbootlf.700 of=/dev/rmt/0mn bs=2k
4.     Again, sync the disk:
# sync;sync
5.     Use dd to append the Root file system:
# dd if=/dev/rdsk/c201d6s0 of=/dev/rmt/0m bs=64k
The first dd puts a boot area on the tape, making it a bootable image. Once the boot image is on tape, the tape is not rewound. The next dd appends an image of the SCSI disk at address 6 to the tape. Be sure to use the appropriate disk device file. This whole process takes about one hour for a 660MB HP2213 disk.
Once created, the tape can be used to completely restore the disk:
1.     Insert the tape into the DDS tape drive.
2.     Boot to ISL from the tape:
BOOT_ADMIN> b scsi.addr_of_dds ipl
3.     Enter the following in response to the ISL prompt:
ISL> hpux restore disc(scsi.6.0;0)
7.3 Interoperability Issues
Both AIX and HP-UX support the following standard UNIX utilities, each of whose behavior is nearly identical: dumprdumptarcpiodd, and pax. For doing interplatform backups it is advisable to use one of these commands. HP-UX has the standard utility mt; AIX does not but has the rough equivalent in tctl. AIX does not have the tcio command; instead you have to use dd to buffer data when using redirection. The following table summarizes the backup commands for both operating systems.
Backup Utilities Comparison
The following backup commands have been tested on an HP Model 712 and an IBM RS/6000 355.
Using tar to Copy Files from One System to the Other
You can tar a directory tree or a set of files on the HP, ftp the tar file to the RS/6000, and then unpack the file. For example,
Back up the .vue directory to the /tmp/vue.tar archive file:
# tar -cvf /tmp/vue.tar .vue
Copy the file to the RS/6000:
# ftp RS_node
ftp> put /tmp/vue.tar
Login to the RS/6000:
# remsh RS_node
Restore the files from the /tmp/vue.tar archive:
# tar -xvf /tmp/vue.tar
Of course you can use rcp instead of ftp or rlogin or telnet instead of remsh.
A quicker way to do the above is to use remsh and tar in a pipe, but this works only if your .rhosts is set up properly. For example:
# tar -cvf - .vue | remsh RS_node "(cd /tmp; tar -xvf -)"
This command writes the archive of .vue to standard output (the - option) and sends that output to the tar command on the RS/6000 and puts the unpacked files in /tmp.
Instead of remsh AIX has the rsh command. So the above command going the other direction would look something like:
# tar -cvf - doc | rsh HP_node "(cd /usr/local; tar -xvf -)"
This backs up the subdirectory doc and puts it in the HP's /usr/local directory.
You can also use cpio to copy a directory tree:
# find /.vue -print | cpio -ov | remsh RS_node cpio -idv
This copies the HP's /.vue directory to the RS/6000's / directory.
Using Another System's Tape Drive
For the following examples, the HP Model 712 had a 4mm DAT drive, the RS an 5GB 8mm tape drive. Since these involve using the network and the dd command, make sure the RS/6000 tape drive is set to variable-length blocks.
Examples using tar
Archive the HP's .vue directory onto the 8mm tape drive of the RS/6000:
# tar cvf - /.vue | remsh RS_node dd of=/dev/rmt0 obs=20b
Archive the RS/6000's /doc directory onto the 4mm DAT drive of the HP:
# tar cvf - /doc | rsh HP_node dd of=/dev/rmt/c201d3m obs=20b
Restore the contents of the tape from an RS/6000 drive to the HP:
# remsh RS_node dd if=/dev/rmt0 bs=20b | tar xvfb - 20
Restore the contents of the tape from an HP drive to the RS/6000:
# rsh HP_node dd if=/dev/rmt/c201d3m bs=20b | tar -xvBf -
Examples using backup and restore
These examples are for using the RS/6000's backup command to write to an HP drive. HP-UX cannot restore the files; you have to use AIX.
# find /usr/local/data -print | backup -f - -iv | rsh HP_node dd\ of=/dev/rmt/c201d3m
# rsh HP_node dd if=/dev/rmt/c201d3m | restore -xvf -
Examples using pax
pax, short for portable archive exchange, is great for interplatform backups and restores. Both HP-UX and AIX support the pax command, which looks and acts the same on both systems. By default pax writes to standard output (with the -w option) and reads from standard input (with the -r option). Without the -w or -r option pax simply lists the files contained in an archive.
To archive verbosely the /doc directory from the HP to the RS/6000's tape drive:
# pax -wv /doc | remsh RS_node dd of=/dev/rmt0 obs=64k
To read the above archive tape from the RS/6000:
# dd if=/dev/rmt0 ibs=64k | pax
To restore the contents of the above archive on the RS/6000 from the RS/6000:
# dd if=/dev/rmt0 ibs=64k | pax -r
To copy the directory /usr/local/data from the RS/6000 to the HP using the network:
# pax -wv /usr/local/data | rsh HP_node pax -rv
General Tips
Variable block mode depends on the block size specified by the command itself (for example, the -b option in tar). Because other systems, including HP, often break up larger reads and writes into approximately 64K chunks, it is a good idea to keep your block sizes at no more than that amount.
Remember that if you are using an IBM 8mm tape drive, set the fixed block size to 1024 to avoid wasting tape. For example,
chdev -l rmt0 -a block_size=1024
7.4 Summary
AIX and HP-UX support a wide variety of backup utilities, ranging from the proprietary to the universal. AIX's proprietary commands are backup and restore, based on dump and restorebackup can do a file system dump or it can back up individual files. restore can restore an entire file system or individual files as well. AIX writes to tape either fixed-length blocks or variable-length blocks. You can configure the system for one or the other by using SMIT or the chdev command. For network backups or when the application determines block size, it is best to set the block size to variable length. Instead of having the mt command, AIX has the tctlcommand, which is very similar to mt.
HP-UX's proprietary backup utilities are fbackup and frecover. These commands resemble backup and restore to an extent but are much more versatile, particularly in the area of graph files and network backups. To improve performance while writing to a cartridge tape, HP-UX supplies the tcio command.
Interplatform backups are possible using standard UNIX utilities like tarcpiodd, and pax, and the networking commands of rsh (AIX) and remsh (HP-UX). These commands need the appropriate .rhosts entries in order to work properly. If you write to another system's tape drive, you must use a common command like tar if you want to eventually restore to the other system. But, for example, if you use backup to write to an HP-UX machine's tape drive, you can only restore your files to an AIX system. For best portability, use pax, the portable archive exchange command.