~~~~~~~~~~~~~~~~~~~~~~~ du ~~~~~~~~~~~~~~~~~~~~~~~
Below command will show the disk space used by first level of each folder in RH Linux. It is a good way to find which folder fills up the disk space.
$ du -h --max-depth=1
To get size of current directory:
$ du -sh * <== List space used by each file & folder under it
$ du -sh (or, $ du -sk ) <== Get the total space used in current location
$ du -k . <== List all details under current location. Not useful.
But, file size reported by "du -sh filename.txt" is much less than the size from "ls -al filename.txt". The difference is big for a large file. A good thing is both "du --apparent-size -m filename.txt" and "ls -lh filename.txt" agree on file size on my RHEL servers.
~~~~~~~~~~~~~~~~~~~~~ find big files ~~~~~~~~~~~~~~~~~
Find the big files under a directory in Linux, which is useful when disk space is filling up:
$
For example
$ find /u01/app -type f -size +1000000 -exec ls -alh {} \;
-rw-rw-r-- 1 ebs oracle 953M Aug 9 2011 /path/to/ftp/out/gl_journal_to_adam.txt
$ du -k . <== List all details under current location. Not useful.
But, file size reported by "du -sh filename.txt" is much less than the size from "ls -al filename.txt". The difference is big for a large file. A good thing is both "du --apparent-size -m filename.txt" and "ls -lh filename.txt" agree on file size on my RHEL servers.
~~~~~~~~~~~~~~~~~~~~~ find big files ~~~~~~~~~~~~~~~~~
Find the big files under a directory in Linux, which is useful when disk space is filling up:
$
find /home -size +1000000 -exec ls -alh {} \;For example
$ find /u01/app -type f -size +1000000 -exec ls -alh {} \;
-rw-rw-r-- 1 ebs oracle 953M Aug 9 2011 /path/to/ftp/out/gl_journal_to_adam.txt
-rw-r----- 1 ebs oracle 1.8G Sep 27 2013 /path/to/ora/10.1.3/Apache/ssl_engine_log
-rw-r--r-- 1 ebs oracle 8.3G Mar 30 13:24 /path/to/ora/10.1.3/j2ee/oacore/oacore_default_group_1/application.log
You can search for exact file size, or just for bigger (+) or smaller (–) files. For example all bigger than 512k files would be found with command:
$
units:
b – for 512-byte blocks (this is the default if no suffix is used)
c – for bytes
w – for two-byte words
k – for Kilobytes (units of 1024 bytes)
M – for Megabytes (units of 1048576 bytes)
G – for Gigabytes (units of 1073741824 bytes)
~~~~~~~~~~~~~~~~~~~~ find the total files ~~~~~~~~~~~~~~~~~~~
Find the total number of files under a directory:
$
$
~~~~~~~~~~~~~~~~~~~~ find then sort ~~~~~~~~~~~~~~~~~~~~~~
Find and sort by timestamp:
$ find /path/. -name '*.*' -printf "%T+\t%s\t%p\n" | sort | more
%T+ - modification time (and date)
%s - size
%p - path of file
~~~~~~~~~~~~~~~~~~~~ find then replace ~~~~~~~~~~~~~~~~~~~
Use below line to replace wrong file owner under a directory:
# find /u01 -user wrongUserID -print | wc -l
-rw-r--r-- 1 ebs oracle 8.3G Mar 30 13:24 /path/to/ora/10.1.3/j2ee/oacore/oacore_default_group_1/application.log
You can search for exact file size, or just for bigger (+) or smaller (–) files. For example all bigger than 512k files would be found with command:
$
find /home -type f -size +512k -exec ls -lh {} \; units:
b – for 512-byte blocks (this is the default if no suffix is used)
c – for bytes
w – for two-byte words
k – for Kilobytes (units of 1024 bytes)
M – for Megabytes (units of 1048576 bytes)
G – for Gigabytes (units of 1073741824 bytes)
~~~~~~~~~~~~~~~~~~~~ find the total files ~~~~~~~~~~~~~~~~~~~
Find the total number of files under a directory:
$
find folder_name -type f | wc -l$
find folder_name -maxdepth 1 -type f | wc -l <== to exclude subdirectories~~~~~~~~~~~~~~~~~~~~ find then sort ~~~~~~~~~~~~~~~~~~~~~~
Find and sort by timestamp:
$ find /path/. -name '*.*' -printf "%T+\t%s\t%p\n" | sort | more
%T+ - modification time (and date)
%s - size
%p - path of file
~~~~~~~~~~~~~~~~~~~~ find then replace ~~~~~~~~~~~~~~~~~~~
Use below line to replace wrong file owner under a directory:
# find /u01 -user wrongUserID -print | wc -l
# find /u01 -user wrongUserID -exec chown rightUserID:userGroup {} \;
To find/list all files under current directory tree that is not owned by you (applMgr):
To find/list all files under current directory tree that is not owned by you (applMgr):
$ find . ! -user applMgr
$ find . ! -user applMgr -exec ls -altr {} \;
~~~~~~~~~~~~~~~~~~~ find then delete ~~~~~~~~~~~~~~~~~~~~
Use find to delete files:
One day, CM log file and Output file can not be opened by browser with error "Authentication failed." I found the disk space was full with 11GB of Apache folder. Use "find" to delete old files (-r will also delete sub-folders):
$ cd $LOG_HOME/ora/10.1.3/Apache
$ find *.* -type f -mtime +7 -exec rm -rf {} \;
NOTES: If there is a huge pile of files, above command may get error. Try a different way by placing * inside the single quotes so that it is used as a matching wildcard and not for shell filename expansion:
$ find . -name '*.tmp' | wc -l
$ find . -name '*.tmp' -mtime +7 -exec rm -f {} \;
Below first line worked for me (in cron also) to delete files from a folder with more than 15K files. But the 2nd line failed.
$ /usr/bin/find /u01/path/utl_dir/. -name '*.tmp' -mtime +60 -exec rm -f {} \;
$ /usr/bin/find /u01/path/utl_dir/*.tmp -mtime +60 -exec rm -f {} \;
-ksh: /usr/bin/find: /usr/bin/find: cannot execute [Argument list too long]
~~~~~~~~~~~~~~~~~~~~~ zip ~~~~~~~~~~~~~~~~~~~~~~~~~~
To Zip up a folder, including all subdirectories:
$ cd /path/to/parent_folderName <= go to the upper folder first
$ zip -r file_name.zip folderName
this is useful when use "admrgpch -s source_path -d target_path -manifest manifest.txt" to merge patches and then copy the merged folder to different node.
~~~~~~~~~~~~~~~~~~ grep / egrep ~~~~~~~~~~~~~~~~~~~~~~~
To find multiple words in a text file:
$ /bin/grep -i -e 'drop ' -e 'alter ' -e 'grant ' -e 'commit ' ascii_file.sql
$ egrep -i 'fail|error' long_file_name.log
~~~~~~~~~~~~~~~~~~~ lsof ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lsof to find open files. it is useful when the system acts strangely
$ lsof /home | wc -l
$ lsof | grep deleted | wc -l (files in being deleted queue)
Use find to delete files:
One day, CM log file and Output file can not be opened by browser with error "Authentication failed." I found the disk space was full with 11GB of Apache folder. Use "find" to delete old files (-r will also delete sub-folders):
$ cd $LOG_HOME/ora/10.1.3/Apache
$ find *.* -type f -mtime +7 -exec rm -rf {} \;
NOTES: If there is a huge pile of files, above command may get error. Try a different way by placing * inside the single quotes so that it is used as a matching wildcard and not for shell filename expansion:
$ find . -name '*.tmp' | wc -l
$ find . -name '*.tmp' -mtime +7 -exec rm -f {} \;
Below first line worked for me (in cron also) to delete files from a folder with more than 15K files. But the 2nd line failed.
$ /usr/bin/find /u01/path/utl_dir/. -name '*.tmp' -mtime +60 -exec rm -f {} \;
$ /usr/bin/find /u01/path/utl_dir/*.tmp -mtime +60 -exec rm -f {} \;
-ksh: /usr/bin/find: /usr/bin/find: cannot execute [Argument list too long]
~~~~~~~~~~~~~~~~~~~~~ zip ~~~~~~~~~~~~~~~~~~~~~~~~~~
To Zip up a folder, including all subdirectories:
$ cd /path/to/parent_folderName <= go to the upper folder first
$ zip -r file_name.zip folderName
this is useful when use "admrgpch -s source_path -d target_path -manifest manifest.txt" to merge patches and then copy the merged folder to different node.
~~~~~~~~~~~~~~~~~~ grep / egrep ~~~~~~~~~~~~~~~~~~~~~~~
To find multiple words in a text file:
$ /bin/grep -i -e 'drop ' -e 'alter ' -e 'grant ' -e 'commit ' ascii_file.sql
$ egrep -i 'fail|error' long_file_name.log
~~~~~~~~~~~~~~~~~~~ lsof ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lsof to find open files. it is useful when the system acts strangely
$ lsof /home | wc -l
$ lsof | grep deleted | wc -l (files in being deleted queue)
~~~~~~~~~~~~~~~~~~ port ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To find what is using a port in Linux:
To find what is using a port in Linux:
Below line will tell if a port is used or not on the server:
$ netstat -tuanp | grep 6230
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 167.69.109.82:6230 0.0.0.0:* LISTEN 31876/opmn
Then the process owner (or root) can use below line to see more details on using the port:
$ lsof -i :6230
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
opmn 31876 ebsdev 9u IPv4 42886593 0t0 TCP server3d.domain.com:6230 (LISTEN)
Below line may also works (Note: "lsof" may show result only to root or process owner):
$ ls -l /proc/31876/exe <= use the PID from above line
lrwxrwxrwx 1 ebiz user 0 Mar 5 11:19 /proc/31876/exe -> /u05/app/EBSDEV/apps/tech_st/10.1.3/opmn/bin/opmn
NOTES: even a port is free and available on a server, it can be blocked by firewall and such to prevent access from other servers.
~~~~~~~~~~~~~~~~~ screen ~~~~~~~~~~~~~~~~~~~~~~~
$ ls -l /proc/31876/exe <= use the PID from above line
lrwxrwxrwx 1 ebiz user 0 Mar 5 11:19 /proc/31876/exe -> /u05/app/EBSDEV/apps/tech_st/10.1.3/opmn/bin/opmn
NOTES: even a port is free and available on a server, it can be blocked by firewall and such to prevent access from other servers.
~~~~~~~~~~~~~~~~~ screen ~~~~~~~~~~~~~~~~~~~~~~~
First of all, make sure "screen" is installed (in /usr/bin).
To start a "screen" session for a long-run job:
$ screen
$ adop phase=fs_clone
... ...
To list running screen/background session:
$ screen -list (Or, $ screen -l)
There are screens on:
1653.pts-5.server2d (Attached)
23515.pts-3.server2d (Attached)
2 Sockets in /var/run/screen/S-userID.
To re-attach a session:
$ screen -r -d 23515
Session ID: 16
Node: server2d
Phase: fs_clone
Phase: fs_clone
Log: $ADOP_LOG_HOME/16/xxxxxx_201956/adop.log
... ...
~~~~~~~~~~~~~~~~~ rsync ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To copy a folder to another folder on the SAME server and preserve exactly all attributes. For example, in EBS R12.2, to synchronize custom folder xxap from PATCH file system to RUN file system:
$ rsync -a --delete $PATCH_BASE/EBSapps/appl/xxap $RUN_BASE/EBSapps/appl
Notes: add rsync -a --delete %s_current_base%/EBSapps/appl/xxap %s_other_base%/EBSapps/appl to file $APPL_TOP_NE/ad/custom/adop_sync.drv
If contents of a folder are static, add rsync -zr %s_current_base%/EBSapps/appl/xxe4a %s_other_base%/EBSapps/appl to file $APPL_TOP_NE/ad/custom/adop_sync.drv
The option -a is very powerful, it will copy the directory and all its sub-directories recursively, and preserve symbolic links, modification times, group, ownership, and permissions.
-a option is a combination of a bunch of other options:
-r Recurse into directories
-l Copy symlinks as symlinks
-p Preserve file permissions
-t Preserve modification times
-g Preserve group
-o Preserve owner
-D Preserve device and special files
"-- delete" will DELETE files in Target folder but not in Source folder.
Below line will copy the folder structure with same permissions to current location, without copying files:
$ cd /path/to/target
$ rsync -av -f"+ */" -f"- *" /path1/to/source .
$ cd /path/to/target
$ rsync -av -f"+ */" -f"- *" /path1/to/source .
Copy a folder to a different server (and preserve exactly all attributes):
$ cd /to/path/source_FOLDER
$ rsync --progress -avze ssh applmgr@nodeName:/to/path/target_FOLDER
~~~~~~~~~~~~~~~~~~ Stickybit ~~~~~~~~~~~~~~~~~~~~~
If your ID owns /path/to/LOG directory and you want to allow users in other Groups to rename the files in that directory, you have to remove Stickybit on LOG directory (by chmod -t) if it is enabled, because enabled Stickybit makes other users unable to rename/delete the files under LOG directory.
$ ls -al /path/to/LOG
drwxrwxrwt 10 user group 134 Aug 14 09:57
$ chmod -t /path/to/LOG # remove Stickybit
$ ls -al /path/to/LOG
drwxrwxrwx 10 user group 134 Aug 14 09:57
Right permission is needed for others to write/modify your file. To get 664 permissions (-rw-rw-r--) on all files created by your ID, add one line to .bashrc or .kshrc or .profile of your OS ID, or add the line to the shell script for generating the file:
set umask to 0002 (or 002 ?)
or
umask u=rwx,g=rwx,o=rx
~~~~~~~~~~~~~~~~~ chmod -R ~~~~~~~~~~~~~~~~~~~~
Run "chmod -R" to change permission on a folder and all subfolders & files under it. For example, to make a folder and all subfolders & all files under it "readable":
$ chmod -R +r /path/to/LOG
- To list all folders (sub-directories called "fs_clone") under a location:
$ find . -name fs_clone -type d -print
- Get the list of directories. Below were tested in K Shell
$ echo $SHELL
/bin/ksh
1.
$ find /u01/app/patches/* -prune -type d (Or go to the directory first, $ find * -prune -type d )
/u01/app/patches/11902981
/u01/app/patches/13006289
/u01/app/patches/16367827
Note: if just want the list of files, use "find * -prune -type f"
2. -- List directories older than two days
$ find * -prune -type d -mtime +2
3.
$ ls -l |egrep "^drw" |awk '{print $9}'
-- Loop through the list to do whatever you want
$ for x in `ls -l |egrep "^drw" |awk '{print $9}'`
do
echo $x;
done
4.
-- Note: FS is used to define the delimiter in AWK
The default is defined by OS env variable $FS
$ ls -1d */ | awk 'BEGIN {FS="/"};{print $1}'
