Working with AIX filesystem PART 3

Today we shall see something about managing file-systems in IBM AIX.

Managing File Systems

Objective:

•File system integrity
•Manage/Monitor
–File system growth
–File growth
–File system space usage

How/Why Space Management ?


How ?

•Find the causes
•keep track of growing files
•check file system space usage
•check disk space

Explanation:


•If a File System reaches its full capacity, that means it cannot take any more data onto it. The filesystem size in these cases has to be increased because the file system size doesn‟t automatically increase.

•Our duty as system administrators is to use commands and techniques to find the causes for the increase in the file system size. This may involve tasks like keeping track of the growing files in the file system. Keep checking the file system space usage and decide whether to increase the size or not.

•The files that could keep growing at high speed are:

a./var/adm/wtmp-> Which keeps track of successful login information
b./etc/security/failedlogin-> which keeps track of failed login attempts.
c./var/spool-> directory which keeps cronentries, mails etc
d.$HOME/smit.log and smit.script-> smitrelated log files
e.$HOME/websm.log and websm.script-> WSM related log files
f./var/adm/sulog-> Keeps track of sucommand usage.

•Many of the above files are required files as they contain very useful information of activities going performed on the system. Completely removing them of may not be a recommended strategy but keeping eye on their increasing size and timely cropping them and removing old entries would be the maintenance task that could be done by the Administrators.

Free Disk Space

Explanation:


•You can use the dfcommand to display useful information about the statistics pertaining to the free disk information. The output of this command would show all the file systems, their sizes and out of the total size, how much is used and how much is free.

•Please refer to the man pages on df command to look at some useful options that you could use with df.

•There is also a very useful command called skulker.The skulker command is under /usr/sbin.

•The skulker command can be used to clean up the file systems by removing unwanted or old files. You can modify this skulker shell script by using an editor and configure it according to your environment.

Disk Usage

Explanation:


•The Disk Usage command (du) can be used to list the files and how many number of data blocks are utilized by them.

•If you look at the above example, the du command is giving the information of the file name, and how many number of 512 byte blocks has been allocated to that particular file. So by looking at this output you can make out as to which files are using more data blocks and take actions if required.

•du command comes with lots of very useful options like –x , -k etc. If you want more details on the various options that can be used with the du command, please look at the man pages for the same.

Control Growing Files

•/var/adm/wtmp
•/etc/security/failedlogin
•/var/adm/sulog
•/var/spool/*/*
•$HOME/smit.log
•$HOME/smit.script
•$HOME/websm.log
•$HOME/websm.script

The skulker Command

•The skulker command cleans up file systems by removing unwanted or obsolete files

•Candidate files include:
–Files older than a selected age
–Files in the /tmp directory
–a.outfiles
–corefiles
–ed.hupfiles

•skulker is normally invoked daily by the cron command as part of the crontab file of the root user

•Modify the skulkershell script to suit local needs for the removal of files

Listing Disk Usage

•The ducommand can be used to list the number of blocks used by a file or a directory

# du /home | sort -r -n
624 /home
392 /home/fred
98 /home/tom
54 /home/mary
52 /home/liz
23 /home/suzy
2 /home/guest
1 /home/steve

•To view individual file sizes, use the ls -l command.

Fragmentation Considerations

Without fragmentation
File size = 2000 bytes

With fragmentation
File size = 2000 bytes
Fragment size = 1024 bytes

Considerations to be made:
Disk space allocation
Disk space utilization
I/O activity
Free space fragmentation
Fragment allocation map

Explanation:

•In AIX the filesystem is broken into blocks called as data blocks. These data blocks by default are 4 kb in size i.e. 4096 bytes in size.

•Lets imagine you have a file called file1 which is about 1900 bytes. To store this file in the filesystem, you will require a data block (4 kb). That means file1 occupies 1900 bytes of the data block and the rest of the space in the 4 kb data block is empty. But even though rest of the space is empty, this space cannot be utilized by the file system to allocate it any other file. So in these circumstances, potentially you could have more amount of unused or wasted space.

•Here is where we can get an intro to the concept of  “Fragmentation”.

•Fragmentation is a concept wherein a single data block can be broken logically into smaller fragments of size ranging from 512 bytes to 4 kb. Lets say I choose a fragment size of 1kb, that means the data block of 4 kb size is broken into 4 fragments.This would ensure that if there was a small file lets say about 1900 bytes, to store this particular file, I would potentially use up only 2 fragments of the data block and the other two fragments can be allocated to files if necessary. Thus reducing the amount of unused and wasted space in your filesystems.

Defragmenting a File System

•The defragfs command increases a file system's contiguous free space

•The file system must be mounted

defragfs [-q | -r | -s] filesystem

Options:

-q Reports the current state of the file system

-r Reports the current state of the file system and the state that would result if the defragfs command is run without either -q, -r or -s

-s Gives short report regarding the current stateof the file system

Explanation:

•Although fragmenting decreases the amount of unused or wasted space, this can actually cause some performance related problems also.

•For example: Lets imagine that the fragment size is 4 kb. There is file of 4 kb size. So 1 data block (fragment) would be assigned to the file. If I want to read this file there would be 1 I/O operating that needs to be done to read the content of the file (disk I/O happens in terms of fragment size).

•Lets say I have the same 4 kb file, but the fragment size instead of 4 kb, I‟ve changed it to 1 kb. That means to allocate space to this particular file I may require 4 1kb blocks. That means if I want to read the same file now there would be 4 disk I/O operations that needs to happen. Which would be time consuming.

•Defragmentation of the filesystemincreases the file systems contigous free space by rearranging the fragments in such a way that files get more contigousspace for allocation.

•The command:

•# /usr/bin/defragfs (-q –r ) filesystem

•can be used to report the current status of the file system and also report what would be the state of the file system after defragmentation is done.

•The file system should be mounted when you run this command.

Verify a File System

•Command syntax:

fsck [-p | -y | -n] [-f] [ file system ]

•Checks journal log

•Checks inodes, indirect blocks, data blocks, free lists

•If no file system name is specified, the fsck command checks all file systems which have the check=true attribute set in the /etc/filesystems

•Orphan files are placed in the lost+found directory

•Unmount the file system before running fsck

Complete explanation:

•File system check or fsckcan be used to verify a file system and check for its consistency. It basically checks the journal log to determine if any activity was pending, it checks the inodes, direct and indirect data block against the free lists.

•fsck command comes with lots of options:-

•-p-> stands for „preen‟. That means do not ask for any confirmation, do any minor changes required. Some people also call this option “silent” that is don‟t disturb me….

•-y-> The answer would be yes for all the questions asked during fsck

•-n-> The answer would be No for all the questions asked during fsck

•-f-> to specify the file system to conduct fsck.

•If no filesystemname is specified with the fsckcommand, all the file systems that have the option check=truespecified in the configuration file /etc/ filesystemswill be checked.

++++++++++++END++++++++++++++

Working with AIX filesystem PART 2

Objective of the module:
How to create a file system
Change the characteristics of a file system

File Systems

Explanation:

You can use the fastpath#smitfsto get into this menu option in AIX to perform file systems related tasks.
File systems can also be managed using the Web Based System Manager.

Listing file systems

Explanation:


You can use the commandlsfsto get a list of all the file systems.

The output in the above visual shows details about the file system. The various attributes of the output are:
a.Name of the file system
b.Node Name, which is only valid if the file system is a Network file system.
c.Mount Point for the file system.
d.Virtual File System (jfs, jfs2 or cdrfs)
e.Size of the file system
f.Mount options if any, like ro(read only) or rw(read-write)
g.Auto, automatic mounting during system startup or not.

You can also use the fastpath #smitfs to get the same details.

Mounted File Systems listing

Explanation:
The mount command can be used to list all the file system that are at present mounted on your system.
It gives you details about the node (valid only for a NFS file system), the logical device name of the file system, mount point, vfs information, date when it was mounted and mount options if any (readonly, readwrite, log information).
You can also get the same information from SMIT also.

Adding a File System

Explanation:


These are the general steps in SMIT to start with the job of creating a file system.
#smit fs will take you to this menu option. Here you can choose the option,
Add/Change/Show/Delete File systems to start adding a file system.

In the next step it asks you to select the type of file system is JFS, JFS2 or cdrfs or NFS. Lets say you select the Journalled File System.

Adding a file system on a previously created LV

Explanation:


In this step, it asks you to select whether you want to create a file system directly or you want to create the file system on a previously defined logical volume.
Lets say you have already created a logical volume and now you want to just create a file system over it, then the ideal option to select is the second option ie Add a journaledfile system on a previously defined logical volume.
When you select the option, it asks you select the type of Journaledfile System iestandard, Compressed or Large File Enabled File System. Lets say you select to add a standard Journalled File System you’ll be shown the appropriate screen to make selections.


Adding a standard jfs file system on previously defined logical volume

Explanation:


This is the dialog panel to create a file system on an existing logical volume.,
The only mandatory fields here are the logical volume name on which you are planning to create the file system and the mount point.

Adding standard JFS

Explanation:


The picture above shows you how you can create a file system directly. That means you want to create the file system and the logical volume in a single step.

The mandatory fields here are you need to select the volume group name where you are planning to create the file system, specify the size of the file system, and the mount point for your file system.

The name of the logical volume thus created will follow the default naming convention in AIX for naming logical volumes i.e. lv00, lv01,lv02 …..

Add a JFS2 File System on a previously created Logical Volume

Explanation:

The above snap shows how you can create a JFS2 file system on a previously existing logical volume. The options are the same like the JFS file system except for the options of having an inline logwhich is a log placed in the same file system.

Add a JFS2 file system

Explanation:


This picture shows how you can directly create a JFS2 file system that means create a logical volume and the JFS2 file system over it in one single step.

Most of the options are just like JFS ie you need to specify the volume group name where the logical volume will be created, the size of the file system and the mount point. You also have an option of enabling inline log option if required.

Mount a File System

Explanation:


You can use the command mount to mount a file system. This can also be done using smit.
File systems that are defined in the file /etc/filesystems with mount=trueor mount=automatic options will be mounted automatically during system startup.

You can either mount a single file system or a group of file systems.
The syntax for mount is
# mount /dev/lv00 /home/user9
Where /dev/lv00is the file system name and /home/user9is the mount point.

You can also unmountthe file system if not required at present. The syntax for the unmountcommand is:
# unmount /home/user9 ( only the mount point name is enough)
The administrator or members of the security group may issue the mount commands assuming that the user has write permission to the mount point and read permission on the root directory of the file system to be mounted.

Change characteristics of a File System

Explanation:

The chfs command can be used to change the attributes of a file system. The picture above is showing how you can use SMIT to change and see the characteristics of a file system.

Dynamically Shrinking a JFS2 File System

Remove a file system

Explanation:


You can use the command rmfsto remove a file system.
The picture shows how you can use the smit to remove a file system.
The file system must be unmountedfrom the tree structure in order to remove the file system. If some user or some processes are using the file system, you will not be able to remove the file system.

NOTE: rmfscommand will remove any information of the file system from the ODM and the /etc/filesystems. When a file system is removed, the logical volume on which it resides is also removed.


Add a RAM File System


•Create a RAM disk of 4 MB
# mkramdisk 4M
/dev/rramdisk0

•Create a JFS file system on this RAM disk
# mkfs -V jfs /dev/ramdisk0
mkfs: destroy /dev/ramdisk0 (yes)? y

•Create mount point
# mkdir /ramdisk

•Mount RAM file system
# mount -V jfs -o nointegrity /dev/ramdisk0 /ramdisk

Working with AIX filesystem PART 1

Objectives for the module
Understand various components of AIX file system.

File system structure

Explanation:

File system resides on a logical volume. So the size of the file system would be in multiples of logical partitions. Every logical partition would be pointing to a physical partition.
An individual file within a file system would be allocated data blocks which is usually 4 KB in size. In traditional UNIX systems the data block size was 512 bytes.

The first very important component of a file system is called the superblock. The superblock contains information about the file system in terms of the name of the file system, its size, number of inodes in the file system, etc.

The superblock information is very critical and will be utilized by the system whenever the file system has to be mounted and accessed. If the superblock goes corrupt, then the file system will not be able to be mounted also. So a backup copy of the superblock is always written in block 31.

Right after the superblock are the inodes. Inodescontains information of files like file permissions, file type, size, owner, group, created/modified date and time and they also contain the pointers to data block.
Data block contain data. Indirect data block maintain addresses to the actual data block.

Contents of the inode

Explanation:
This is the structure of the inodes. Each file on your system is represented by a single inode. The inode for a file contains information about the file in terms of
>Inode number for the file
>Ownership
>Access Permissions
>Type of the file
>Creation, modification and access times
>Number of links to the file
>Size
>Addresses of data blocks on disk.

Use can use the command ls with the option –i to look at the contents of the inode for that file.

File System Fragmentation

Explanation:
In AIX the filesystemis broken into blocks called as data blocks. These data blocks by default are 4 kb in size i.e. 4096 bytes in size.

Lets imagine you have a file called file1 which is about 1900 bytes. To store this file in the filesystem, you will require a data block (4 kb). That means file1 occupies 1900 bytes of the data block and the rest of the space in the 4 kb data block is empty. But even though rest of the space is empty, this space cannot be utilized by the file system to allocate it any other file. So in these circumstances, potentially you could have more amount of unused or wasted space.

Here is where we can get an intro to the concept of “Fragmentation”.

Fragmentation is a concept wherein a single data block can be broken logically into smaller fragments of size ranging from 512 bytes to 4 kb. Lets say I choose a fragment size of 1kb, that means the data block of 4 kb size is broken into 4 fragments.This would ensure that if there was a small file lets say about 1900 bytes, to store this particular file, I would potentially use up only 2 fragments of the data block and the other two fragments can be allocated to files if necessary. Thus reducing the amount of unused and wasted space in your filesystems.

Number of bytes per inode

For how many number of bytes should a inode be generated ?
for example: nbpi=4096 would create an inode for every 4096 bytes.

Explanation:
In a Journaled File System, when ever a file is created, its respective inode also has to be created. The inode maintains details about the file. In earlier versions of JFs, the number of inodes created for a file system was fixed and it was generated for every 4 KB of disk space of that file system. That means 1024 inodes would be generated. If the data block size also remains at 4 KB, these many inodes was actually enough to cater to the needs of the file system.

But since fragmentation came into picture, wherein instead of sticking to 4 KB data block size, the data block was able to be fragmented in smaller sizes lets say 1024 ie1KB, the need for more number of inodes also increased since there would be potentially more smaller files in the file system. Here is where the nbpi value (number of bytes per inode) can be reduced/changed to suit the requirements. So now nbpi=1024 would mean, please create a inode for every 1024 bytes of the space allocated to this file system.

Allocation Group Size

Explanation:

Allocation group size concept increases the efficiency of the file system. Allocation group is nothing but a logical grouping of related datablocks and the inode into a single group to increase the speed at which the data can be accessed within a filesystem.

The following small table gives you the Allocation Group Size and its respective nbpi values that are supported:
Allocation Group Size                   NBPI
8 MB                              512,1024,2048,4096,8192,16384
16 MB                           1024,2048,4096,8192,16384,32768
32 MB                           2048,4096,8192,16384,32768,65536
64 MB                           4096,8192,16384,32768,65536,131072

Compressed file system

Explanation:
Compressed file system save disk space by allowing a logical block to be stored on the disk in units or fragments smaller than the full block size of 4096 bytes.

Data compression allows all logical blocks of any sized file to be stored as one or more contiguous fragments. On an average, data compression saves disk space by about a factor of 2. There is no support for compression with JFS2 file system.

Recommendations: The root file system must not be compressed. Compression of the /usr file system is not recommended.

Large File Enabled File Systems

                                            File = 132 MB


(1024 * 4 KB blocks) + (1024 * 128 KB blocks) = 132 MB
4 MB                          + 128 MB                          = 132 MB

Explanation:

Before discussing the Large File Enabled File System, Lets take a normal scenario of a file of size 140 MB. The default block size for this file system is 4 KB. That means, to store a file of 140 MB size in 4 KB blocks we would require 140 MB /4 KB = 35840 number of data blocks. That means to read this file the system has to perform 35840 I/O operations.

With Large File Enabled File System, the first 4 MB of the file will be allocated in 4 KB blocks and the rest of it will be allocated in 128 KB blocks. That means for the same file which is 140 MB in size, first 4 MB will be allocated using 4 KB blocks (4MB / 4 KB = 1024 blocks) and rest of the file is 136 MB will be allocated in 128 KB block ( 136 MB / 128 KB=1088 blocks). Totally in a Large File Enabled File System to allocate this file of 140 MB only 1024+1088 ie 2112 data blocks will be required and this calculation itself is an indication how a Large File Enabled File System can bring in performance. ( Only 2112 I/O operations are required to read the file instead of 35840 blocks.

Journal Log

Explanation:

Jfslogs are kept in /dev/hd8. The journal logs are circular logs. The size of the journal log is 1 PP size. The jfslog is used to ensure file system integrity by writing all metadata information to the jfslog immediately.
In journal log, only inode and indirect data block information is stored and not the actual data block.
There is also a new feature called inline log that is introduced with JFS2. This inline log allows you to log directly to the file system.

Difference between JFS and JFS2

Explanation:

JFS2 (Enhanced JournaledFile System)
Is a new file system introduced in AIX5.1. The base for JFS2 is JFS file system. At present JFS2 is the only standard file system supported on Itanium based platform.

Extent Based Allocation:
JFS2 uses extent based allocation. An extent is an address length pair, which identifies the starting block address and the length of the extent in blocks. This allows multiple adjacent blocks to be addressed. The advantage of extend based allocation are high performance and large file size.

In JFS2 the inodes are created dynamically i.e. on fly whereas in JFS there was a fixed number of inodes that was getting created during the creation of the file system. An inode in JFS2 file system is of size 512 bytes when compared to the JFS inode size of 128 bytes.

Migration issues: JFS and JFS2 can co-exist on the same system. But to migrate a JFS into JFS2, take the backup of data on the JFS file system, create a new JFS2 file system, restore the JFS file system backup on the new JFS2 file system.






Incoming search terms:
AIX file system structure
File system in AIX
AIX filesystem
Description of AIX filesystem
JFS and JFS2
Difference between JFS and JFS2
Journal log
Journal log in AIX
Inode in AIX
Allocation group size in AIX
Compressed file system in AIX
step by step create a filesystem on aix