awk : one-liner to select a field

Getting a specific field of a line with awk is really simple. For example :
spaghetti% echo "test1 test2 test3" | awk '{print $2}'
test2

A more “real life” example is as following, which will find in /etc/hosts the IP address of localhost (pick another host if you wish 🙂 ) :
spaghetti% cat /etc/hosts | awk '/localhost/ { print $1;}'
127.0.0.1
::1

If the field separator is not a space, awk will let you change it with the FS variable :
spaghetti% echo "test1:test2:test3" | awk 'BEGIN { FS = ":"} { print $2 }'
test2

This should be enough to get you started !

Note : As suggested by Miljan, there’s a much simpler way to change the Field Separator : the -F option switch. So the example I provided can be rewritten as follows :
spaghetti% echo "test1:test2:test3" | awk -F: '{ print $2 }'
test2

Cool. Thanks Miljan !

sed : replacing a text in a file

To replace a text in a file, you can invoke sed as in the following example :

% cat file.txt | sed -e 's/text/replacement/g' > result.txt

This will change all the occurences of “text” to “replacement” in “file.txt” and output the result in “result.txt”

Note : As suggested by Matthias from adminlife in the comments, if you wanted to do “in place” text replacement (that is modify the file without a temporary file in between), you can do the following :

sed -i ’s/text/replacement/g’ file.txt

For more complicated text manipulations you might consider moving to Perl, but sometimes you don’t need the sledge-hammer 🙂

Linux – RedHat (or CentOS) : update system and packages with yum

In order to keep your system updates (especially important for security fixes) on a RedHat linux system (or CentOS), you can simply perform the following command :
yum update
You’ll then be presented with a list of available updates for your system.

Doing so from time to time will help you to keep your system secure and to get the latest versions of your softwares.

If you wonder how to do the same with a Debian Linux system, check out my previous post about Apt, the Debian package manager.

Unix : Cron daemon, crontabs and where to find them

Cron is a fairly standard daemon which you’ll find on most (if not all) Unix machines. Its purpose is to schedule the execution of commands at a specified time.

Sometime you’ll log at some performance graph (CPU load for example) and find out that every day/week/month/other there is an unexpected peak and you’d like to know why. Of course if this is regular you’ll think of Cron as a good trail to follow.

Configuration files for those scheduled jobs are stored in files which are called “crontabs”. Usually, a man crontab will give you the format of this file.

Now the next question you’ll ask is “where to find all the scheduled jobs ?”

The answer varies greatly following the Unix flavor your using, as well as the cron version or clone is running on the system.

Standard places are :

  • /var/spool/cron : in this directory you’ll find a file for each user who has cron scheduled jobs
  • /etc/crontab : this is a system global crontab. Sometimes this file will reference other directories (or files, depending on the clone of Cron) such as :
    • /etc/cron.hourly : hourly jobs, usually running at the start of a new hour
    • /etc/cron.daily : daily jobs, usually running during the night
    • /etc/cron.weekly : weekly jobs, usually running the night between saturday and sunday
    • /etc/cron.montly : montly jobs, usually running on the 1st of the month
  • Maybe your Cron clone will also allow you to have an /etc/cron.d directory where you’ll find application specific crontabs (for exemple a database might install there a crontab to schedule tables optimizations)

The usual syntax of a crontab is :
* * * * * user command-to-run

The 5 first fields (stars in this example) are for :

  1. minute
  2. hour
  3. day of month
  4. month
  5. day of week

If you put a star, then it means “every”, the example means every minute of every day of every month… you get it ;-). You can put multiple values separated by a coma.

If the minute field was “0,30” then the job would run at those minutes.

Some versions of Cron will allow you to write “*/N” meaning every N minutes / hours / whatever.

You can usually list cron jobs for a user by issuing a crontab -l user (yourself if you don’t specify a username) or edit them with crontab -e user (same remark applies).

There is much more to tell about cron (like about the output of the script being emaild, environment variables and so on), but this would go further than this introductory post … refer to man cron and man crontab to get all the gory details 🙂

Linux LVM : A short intro

If you are running Linux, then you can use LVM (Logical Volume Manager) to get an extra flexibility in the way you allocate your disk space.

Physical disks are wrapped in Physical Volumes (PVs), which are grouped in Volume Groups (VGs). Logical Volumes (LVs) can then be laid over a VG.

So, if you want do manage a disk through LVM, you start by creating a PV for it ( with pvcreate ).

Once this is done, the disk is ready to be used in LVM. You can then choose either to create a VG for this disk alone (with vgcreate), or to add it to an existing VG (with vgextend).

The added space of this disk is now available to be used in the LVs which are laid over this VG either by creating a new LV (with lvcreate) or extending an existing LV (lvextend).

If you decide to extend an existing LV, you’ll probably have to extend the filesystem (i.e : ext2fs, ext3fs, reiserfs …) too with the appropriate command (maybe resize2fs or resize_reiserfs).

References : LVM How-To
Man pages for lvm and related pages

Debian : update a dynamic DNS

A bit of context : I have a bind 9 DNS allowing DNS updates from clients on the LAN (ok this is fairly insecure, but still my LAN is my home LAN composed of 4 machines … let’s say that’s good enough for me ! 🙂 )

The named.conf allows those updates with this config directive in the zone config block :
allow-update {mynet; };

and mynet is defined an acl directive to be my LAN.

Then you can update your DNS with the nsupdate tool with a syntax along the following lines :
spaghetti:~# nsupdate
> server 192.168.0.1
> update delete spaghetti.domain.name A
> update add spaghetti.domain.name 8000 A 192.168.0.103
> send
> quit

I was then looking for a place to hook a code snippet doing that update after an update via DHCP and I found the answer in A dynamic dns update client on Debian with dhcp3-client (many thanks dude !).

Here is the drill down :

  1. install the dhcp3-client package (apt-get install dhcp3-client ): this version has easy to use hooks before and after querying the DHCP server to get network config
  2. you can drop a script which will automate the nsupdate in “/etc/dhcp3/dhclient-enter-hooks.d” and it will get run right after network configuration (see the sample debug script for the variable which are available upon script execution, such as $new_ip_address)

Solaris 10 : installing … and starting SSHD

First of, you’ll have to locate and install the following packages :

  • SUNWsshcu
  • SUNWsshdr
  • SUNWsshdu
  • SUNWsshr
  • SUNWsshu

The two last are the SSH client parts, it doesn’t hurt to install them.

You need to have the server keys generated in /etc/ssh. Those are the 4 files :

  1. ssh_host_dsa
  2. ssh_host_dsa.pub
  3. ssh_host_rsa
  4. ssh_host_rsa.pub

Should they not to be there, you can still generate by issuing the following command : /lib/svc/method/sshd -c.

Finally, you can start the service with svcadm enable sshd.

If this fails then you’ll need to have a peek under the hood … The logs of SMF are located in /var/svc/log and the one of sshd is network-ssh:default.log.

Solaris 10: easily deal with removable media

Solaris provides vold (Volume Management Daemon) which lets you deal easily with removable media such as CDs and DVDs.

This tool is provided by the SUNWvolr and SUNWvolu packages. Once you have found those packages and installed them, accessing to your removable medias becomes a bliss : All you have to do is insert your media, and go in the configured directory (ex : /cdrom).

Those directories are defined in /etc/vold.conf.

NB : maybe you’ll need to mkdir /vol, and maybe you’ll have to manually enable the service (svcadm enable volfs) the first time (and preferably in this order).

Solaris 10 : great doc about package management

I stumbled on the Solaris 10 training & tutorials and it holds a really nice doc about package management in Sun Solaris 10, broken in 2 parts :

  1. Performing Solaris 10 OS Package Administration (part1)
  2. Performing Solaris 10 OS Package Administration (part2)

Amongst all the information, I was especially interested in those which I always forget like how to check package installation integrity (or how to know if files have been tampered with since it was installed) with pkgchk PACKAGE_NAME (eg : pkgchk SUNWcsu) or how to check integrity of a file with pkgchk -p /path/to/file (you’ll get extra information such as which package it was installed from by adding the -l flag).

pkgchk will check both file attributes and contents by default, so it is a great help to investigate problems related to installed packages being tampered with (every sysadmin knows this, either because of manipulation error, or an intruder trying to replace core utilities by trojan horses, backdoors, etc).

Of course you’ll also find the more “trivial stuff” like installing a package (pkgadd -d /path/to/packages PACKAGENAME), removing a package (pkgrm PACKAGENAME) and so on ..

Great resources, really !