If you need an easy way to confirm that a text file is DOS or UNIX formatted (they differ with respect to the end of line character(s) for example) or if you wish to display normally non-printing characters of a text file, you can use the -vET command line switches of the cat utility.

As explained in the man page :

• -v : will use the ^ and M- notation for control and multibytes characters
• -E : will make ends of lines visible
• -T : will make tabulations visible

For example :

1
2
3
4
5

% cat -vET test.txt
a test message for$
showing the^Iuse of   cat -vET$
^I$
that is all$

and compare it to the following plain cat output, without any command line option :

1
2
3
4
5

% cat test.txt     
a test message for
showing the     use of   cat -vET
 
that is all

You can easily tell what blank space is a space or actually a tabulation for example. If you’d happen to see ^M characters popping out at the end of the lines, then you this is actually a DOS text file (which you might need to convert to UNIX style end of lines with an utility such as dos2unix)

Unix 101 : Showing non-printing characters in text files (ex : DOS files) is a post from: Tech@Sakana - A sysadmin's blog

Q: If I chmod 777 a file, can someone delete it ?
A: No. To explain that, let’s first talk about directories. You can imagine a directory as a special file which would have one line for each file contained in that directory. On each such line, there would be the inode number, and the name of the matching file.
Deleting a file would be like removing a line from that special directory file which means that you need write permission on the directory containing the file to do so.

Q: What about that 777 mode /tmp directory ? Won’t someone delete all my files then ?
A: No. /tmp ‘s permissions are not actually 777 but 1777. That “1″ is the sticky bit which materialize by the “t” flag showing up in “ls -l”. That sticky bit means that only the owner of the file is allowed to delete it.

1
2

kattoo@roadrunner ~ % ls -ld /tmp
drwxrwxrwt 10 root root 4096 Dec  8 21:53 /tmp/

Q: Knowing that bit about directories being special files, how would you implement a hard links ?
A: Simply have 2 “special file directories” having a line with the same inode number and different (or same !) file names. There you go, alternate filenames for the same on-disk content.

Q: But then, if I delete a file with a hardlink, the “other” filename will point to no data ?
A: No. The inode contains quite a few informations : dates, permissions, owner, group and such as well as the number of hardlinks. Every time you delete such a hardlink, this counter is decremented. Everytime you create a hardlink, the count is incremented. The actual content of the file is only actually deleted when the count reaches 0.

Q: How about soft links (aka symbolic links aka symlinks) ?
A: A symlink is yet another special kind of file. You can imagine that it’s a file which contains the name of another file. Anyone accessing the symlink will actually access the content of the file which name is in the symlink special file. This name resolution being done in-kernel, it’s mostly transparent for userland processes. Notice that you could place the name of a non-existing file in the symlink (dead link !), which is not possible with a hardlink.

Unix 101 : Filesystem basics & Special files is a post from: Tech@Sakana - A sysadmin's blog

Method 1 : Using xmodmap

This is mostly the usual way of altering the keyboard map. All you have to do edit the ${HOME}/.Xmodmap file like below :

1
2

clear Lock
keycode 66 = Escape

This file is often sourced when your X sessions is opened, but there are numerous bug reports stating that it’s either ignored or overridden by some windows managers.

You might need to source the file by running xmodmap ~/.Xmodmap at some point after the windows manager is started.

Method 2 : Using XKB configuration

If you let Xorg setup your keyboard by consulting HAL, you can remap the caps lock key to escape by having the following policy file in /etc/hal/fdi/policy/keyboard.fdi

1
2
3
4
5
6
7
8
9
10
11
12
13

<?xml version="1.0" encoding="UTF-8"?>
<deviceinfo version="0.2">
        <device>
        <match key="info.capabilities" contains="input.keyboard">
                <match key="info.product" contains="Logitech Logitech USB Keyboard">
                        <merge key="input.x11_driver" type="string">evdev</merge>
                        <merge key="input.x11_options.XkbOptions" type="string">caps:escape</merge>
                        <merge key="input.x11_options.XkbLayout" type="string">us</merge>
                        <merge key="input.x11_options.XkbVariant" type="string">intl</merge>
                </match>
        </match>
        </device>
</deviceinfo>

The caps:escape takes care of setting up caps lock as an escape key.

See more about How to setup Xorg according to HAL

If you don’t use HAL and prefer to setup your keyboard through xorg.conf, it will look like the snippet below :

1
2
3
4
5
6
7
8

Section "InputDevice"
    Identifier "Keyboard1"
    Driver "evdev"
 
    Option "XkbLayout" "us"
    Option "XkbVarian" "intl"
    Option "XKbOptions" "caps:escape"
EndSection

To see the other available options, check /usr/share/X11/xkb/rules/xorg

Further readings :

• The XKB Configuration Guide
• An Unreliable Guide to XKB Configuration

Linux + Xorg : Remapping caps lock key to escape is a post from: Tech@Sakana - A sysadmin's blog

Recall a previous line

• Recall the last line
  ^?View Code SHELL

  1 2 3 4 5 6

  kattoo@roadrunner ~ $ date Sun Mar 28 18:35:12 CEST 2010 kattoo@roadrunner ~ $ !! date Sun Mar 28 18:35:13 CEST 2010 kattoo@roadrunner ~ $

• Recall the n-th last line :
  ^?View Code SHELL

  1 2 3 4 5 6 7 8 9 10

  kattoo@roadrunner ~ $ echo 1 1 kattoo@roadrunner ~ $ echo 2 2 kattoo@roadrunner ~ $ echo 3 3 kattoo@roadrunner ~ $ !-3 echo 1 1 kattoo@roadrunner ~ $

• Recall the n-th line of the history
  ^?View Code SHELL

  1 2 3 4 5 6 7 8 9 10 11 12 13

  kattoo@roadrunner ~ $ history [...] 533 date 534 date 535 echo 1 536 echo 2 537 echo 3 538 echo 1 539 history kattoo@roadrunner ~ $ !536 echo 2 2 kattoo@roadrunner ~ $

Re-using previous command line arguments

• Re-use all the arguments of the previous command
  ^?View Code SHELL

  1 2 3 4 5

  kattoo@roadrunner ~ $ touch test-file.txt kattoo@roadrunner ~ $ scp !* remote:/tmp scp test-file.txt spaghetti:/tmp test-file.txt 100% 0 0.0KB/s 00:00 kattoo@roadrunner ~ $

• Re-use the m-th argument from the n-th previous command line
  ^?View Code SHELL

  1 2 3 4 5 6

  kattoo@roadrunner ~ $ echo 1 2 3 1 2 3 kattoo@roadrunner ~ $ echo !-1:2 echo 2 2 kattoo@roadrunner ~ $

• Re-run the previous command changing occurrences of X by Y
  ^?View Code SHELL

  1 2 3 4 5 6

  kattoo@roadrunner ~ $ echo test test kattoo@roadrunner ~ $ ^test^replaced echo replaced replaced kattoo@roadrunner ~ $

That’s about it… those are what I consider to be the most useful history expansion features of bash.

Do you have any other you find especially useful ? Share them in the comments !

Bash / zsh : Using the history expansion is a post from: Tech@Sakana - A sysadmin's blog

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

login@servername:/> sar
 
AIX servername 3 5 00CFE81B4C00    01/25/10
 
Configuration du système : lcpu=20 ent=8,00
 
00:00:01    %usr    %sys    %wio   %idle   physc   %entc
00:05:01       2       2       0      96    0,40     5,0
00:10:01       3       2       0      94    0,48     6,0
00:15:00       1       1       0      98    0,21     2,6
00:20:01       1       1       0      98    0,23     2,9
00:25:01       2       1       1      97    0,25     3,2
00:30:01       1       1       0      98    0,23     2,9
00:35:00       2       1       1      96    0,31     3,8
00:40:01       1       1       0      98    0,21     2,6
00:45:01       1       1       0      98    0,22     2,7
00:50:00       1       1       0      98    0,23     2,8
00:55:00       1       1       0      98    0,21     2,6
01:00:00       1       1       0      98    0,21     2,6
01:05:00       2       1       0      96    0,35     4,4

The columns are the usual for this type of tools :

1. %usr : the percent of time spent in user land. The higher the more CPU intensive processes you are running.
2. %sys : the percent of time spent in kernel. This is mostly due to I/O intensive processes and/or processes performing a lot of syscalls.
3. %wio : I’ve got no idea about that one … If you’ve got an explanation for this, feel free to hit the comments. I suspect this is the time spent waiting for I/Os to complete, or waiting after a busy device. This could be a sign of I/O (disk, network) bottlenecks
4. %idle : the percent of time spent doing … nothing. The higher this number, the more underused your machine
5. physc : no idea neither !
6. %entc : the number of CPU used to absorb the load

If you’re looking for a top like tool, topas is the standard AIX equivalent.

AIX : Use sar to check cpu usage is a post from: Tech@Sakana - A sysadmin's blog

df command

The df (disk free) command will report how much of each file system is used. You can combine this with with the sort command to get the fullest file systems at the end :

1
2
3
4
5
6
7
8
9
10
11
12

kattoo@roadrunner ~/Downloads $ df | sort -n -k 5,5
Filesystem           1K-blocks      Used Available Use% Mounted on
shm                    2029696         0   2029696   0% /dev/shm
/dev/sda6              1968588      3232   1865356   1% /tmp
udev                     10240       228     10012   3% /dev
/dev/sda12           367628876  28899532 338729344   8% /files
/dev/sda7              1989820    246300   1743520  13% /var
/dev/sda9              1999932    260996   1738936  14% /usr/portage
/dev/sda5               244176     52264    191912  22% /
/dev/sda8              9990188   3161340   6828848  32% /usr
/dev/sda11            99947736  32910756  67036980  33% /files/vservers
/dev/sda10             3989912   1699372   2290540  43% /usr/portage/distfiles

The sort arguments are :

• -n : order numerically
• -k 5,5 : order by the 5th field

du command

When you found out that a specific file system is full, you need to quickly analyze which folders are the biggest.
You can use the du (disk usage) command. This command will report the space used by a directory and sub-directories. I usually run something like :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

kattoo@roadrunner /usr/portage $ du -sk * | sort -n | tail -20
5711	x11-misc
6056	app-misc
6224	dev-ruby
6320	app-dicts
7219	profiles
7428	net-analyzer
7686	app-text
8112	media-plugins
8777	kde-base
8883	media-libs
8943	sys-apps
9764	dev-util
9986	media-sound
10955	dev-libs
11295	dev-python
11479	dev-java
11645	net-misc
17553	dev-perl
112556	metadata
1705976	distfiles

This will show the 20 largest files or directories. du will display the size of each directories and sub-directories by default. The -s switch will make du display only the grand total for each arguments.

According to the results, you might want to go on repeating this command in the sub-directories, until you’ve narrowed down enough to find the culprit.

find command

The file system full error is more often than not due to a faulty script or program which logs are running wild due to a bug. This will usually lead to a huge file somewhere in a directory, hogging all the available space on the file system.

An easy way to spot it is to use the find command. The following snippet for example will search for files bigger than 10MB in the current directory and sub-directories and print the 10 largest ones :

1
2
3
4
5
6
7
8
9
10
11
12

kattoo@roadrunner /usr/portage $ find . -type f -size +10000000c -exec ls -l \{} \; | sort -n -k 5,5 | tail -10
-rw-rw-r-- 1 portage portage 23416703 Oct  1 10:07 ./distfiles/samba-3.0.37.tar.gz
-rw-rw-r-- 1 portage portage 28749072 Jan 18  2008 ./distfiles/extremetuxracer-0.4.tar.gz
-rw-rw-r-- 1 portage portage 46905557 Oct 16 17:20 ./distfiles/firefox-3.5.4.source.tar.bz2
-rw-rw-r-- 1 portage portage 46914620 Dec  2 05:32 ./distfiles/firefox-3.5.6.source.tar.bz2
-rw-rw-r-- 1 portage portage 50380241 Oct  2 11:47 ./distfiles/VirtualBox-3.0.8-53138-Linux_amd64.run
-rw-rw-r-- 1 portage portage 50595281 Nov 17 10:44 ./distfiles/VirtualBox-3.0.12-54655-Linux_amd64.run
-rw-rw-r-- 1 portage portage 59368714 Aug  8 02:10 ./distfiles/gcc-4.3.4.tar.bz2
-rw-rw-r-- 1 portage portage 61494822 Sep 10 00:34 ./distfiles/linux-2.6.31.tar.bz2
-rw-rw-r-- 1 portage portage 125384668 Oct  1 13:58 ./distfiles/qt-x11-opensource-src-4.5.3.tar.gz
-rw-rw-r-- 1 portage portage 314942420 Jun 18  2008 ./distfiles/sauerbraten_2008_06_17_ctf_edition_linux.tar.bz2
kattoo@roadrunner /usr/portage $

I also like to filter out already compressed files, in order to collect the biggest files which I could compress to save some space with something like :

1

find . -type f -size +10000000c \! -name "*.Z" \! -name "*.gz" \! -name "*.bz2" -exec ls -l \{} \;

Tip : Set this as an alias in your profile by adding this to your .bashrc (if you’re using bash … otherwise check your shell documentation) :

1

alias bigfiles='find . -type f -size +10000000c \! -name "*.Z" \! -name "*.gz" \! -name "*.bz2" -exec ls -l \{} \; | sort -n -k 5,5 | tail -10'

This will give you the 10 largest not-yet-compressed files in a single handy command.

Caveat

“Too many arguments” error :

If you are working with directory holding an important number of files, you might get a “Too many arguments” error when using a star expansion (like in the du -sk * example above).

This is actually not the du command complaining, but the shell which gets too much data when expanding the star. When this happens, you are usually better off using the find command as explained above.

Another possibility is to pipe the output of find to the xargs command. Basically xargs will take everything on the standard input and give it as arguments to the specified command.

Deleting / compressing a file which is still open
Also beware of this : if you delete or compress a file which is still being open by a process, then the space this file use won’t get freed before the process actually closes the file (I’ll explain why in a different post, this is an interesting topic on its own ).

If you deleted or compressed the file (so basically the big file disappeared or was replaced by a compressed version), but the space doesn’t get freed (which you can check with df), then you can bet that a process is still holding the file open. You can spot this with tools like lsof or fuser. Those tools vary greatly according to which variant of Unix you’re running. On IBM’s AIX, fuser had a handy -d option to spot files on the file system which have a link count of 0 and it will report the PID of the attached processes.

Better check those tools’ man pages before you run into this situation !

A word to the wise

Those recipes will help you to find a way out when you are already facing a file system full problem. The best is of course to avoid them in the first place … The following ideas are a good start :

• Set alerts to have an early warning and let you deal with it before applications start crashing. Tools like Nagios would be your friends here, but home made scripts running from cron and sending emails might be enough.
• Check the trends : You can use tools like cacti to graph the space occupation of your file systems over the time. This will let you anticipate when you’ll need to add more disks or if your log rotation and/or file archiving policies are adequate.

More ideas ? Tips to share ? Hit the comments !

Linux / Unix : Disk usage and identifying biggest files is a post from: Tech@Sakana - A sysadmin's blog

1
2

roadrunner ~ # equery belongs /etc/init.d/samba [ Searching for file(s) /etc/init.d/samba in *... ]
net-fs/samba-3.0.37 (/etc/init.d/samba)

equery is part of the gentoolkit package, which provides useful tools to maintain/investigate your gentoo packages.

1
2
3

roadrunner ~ # equery belongs /usr/bin/revdep-rebuild 
[ Searching for file(s) /usr/bin/revdep-rebuild in *... ]
app-portage/gentoolkit-0.2.4.5 (/usr/bin/revdep-rebuild)

Gentoo Linux / Portage : How to know which package provided an installed file is a post from: Tech@Sakana - A sysadmin's blog

1. Clone the disk image with a similar command line (you will have to adjust your paths) :
   ^?View Code TEXT

   1 2 3 4 5 6 7 8

   kattoo@roadrunner /files/toSave/home/kattoo $ VBoxManage clonehd /files/vservers/{VM1/disk1.vdi,VM2/disk2.vdi} VirtualBox Command Line Management Interface Version 3.0.10 (C) 2005-2009 Sun Microsystems, Inc. All rights reserved.   0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100% Clone hard disk created in format 'VDI'. UUID: 84956ae4-6c85-41cf-aa23-a265029d99a8 kattoo@roadrunner /files/toSave/home/kattoo $

2. Setup a new virtual machine through the virtualbox GUI or otherwise, with the same parameters as the original one and attach this new disk

Note : This command doesn’t take care of potential snapshots you could have on the source disk image. It will only clone the base image, so you might have to merge the snapshots into the base image before cloning if you want those to be integrated in the copy.

References :

• VirtualBox manual : vboxmanage clonehd

VirtualBox : Cloning a virtual hard disk is a post from: Tech@Sakana - A sysadmin's blog

Step 1 : install the packages

(This step is optional if you already have them installed on the machine)

1
2
3
4
5
6

kattoo@backup3:~$ pfexec pkg install SUNWsmbskr
DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  2/2       35/35      2.0/2.0 
 
PHASE                                        ACTIONS
Install Phase                                  90/90

At this point, you’ll need to perform a reboot (if you can’t afford that, check in the references for a link to the full procedure).

Step 2 : Join a WORKGROUP

1
2
3
4
5

kattoo@backup3:~$ pfexec smbadm join -w SAKANA
After joining SAKANA the smb service will be restarted automatically.
Would you like to continue? [no]: yes
Successfully joined SAKANA
kattoo@backup3:~$

Step 3 : Set passwords for users who will access the shares

Add the following line into /etc/pam.conf :

1
2

kattoo@backup3:~$ grep smb /etc/pam.conf 
other 	password required 	pam_smb_passwd.so.1 nowarn

Now reset the passwords for the users who will be access the shares :

1
2
3
4

kattoo@backup3:~$ pfexec passwd kattoo
New Password: 
Re-enter new Password: 
passwd: password successfully changed for kattoo

Step 4 : Share the filesystems

You can then share any zfs filesystem with the following :

1

kattoo@backup3:~$ pfexec zfs set sharesmb=on backup

To check what is exported, you can do this :

1
2
3
4
5
6
7
8
9
10

kattoo@backup3:~$ sharemgr show -pv
default nfs=()
zfs
    zfs/backup smb=()
	  backup=/backup
	  backup_roadrunner=/backup/roadrunner
	  backup_thekla=/backup/thekla
    zfs/storage smb=()
	  storage=/storage
kattoo@backup3:~$

Here, the backup_roadrunner and backup_thekla were automatically exported because the property sharesmb was inherited from backup.

References :

• Genunix : Getting Started With the Solaris CIFS Service

OpenSolaris : Exporting a ZFS filesystem with CIFS is a post from: Tech@Sakana - A sysadmin's blog

If you still want to do that, follow those instructions :

Change the default packages publisher

1

stephane@backup3:~$ pfexec pkg set-publisher -O http://pkg.opensolaris.org/dev opensolaris.org

Update pkg

1
2
3
4
5
6
7
8

stephane@backup3:~$ pfexec pkg install SUNWipkg
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    3/3     113/113     0.49/0.49 
 
PHASE                                        ACTIONS
Removal Phase                                  28/28 
Install Phase                                  12/12
Update Phase                                 227/227

Finally, perform the update of the packages to the /dev branch

1
2
3
4

stephane@backup3:~$ pfexec  pkg image-update
DOWNLOAD                                    PKGS       FILES     XFER (MB)
FSWxorg-fonts-core                         2/666   872/44604   8.20/723.02 
[...]

This might take a while !

References :

• OpenSolaris Development Release Packaging Repository

OpenSolaris : Switching to the /dev development branch is a post from: Tech@Sakana - A sysadmin's blog