Random Wisdom

Software

Processing files using ‘find’

by on Mar.26, 2008, under How To ..., Linux, Software

In its most basic form, find is often used to locate files that are subsequently piped through a complex set of commands for processing. However, this particular method is easily broken by files that contain spaces in their names.

This is where the ‘exec’ option provided by find comes in handy. From the man-page:

-exec command ;
       Execute  command;  true  if 0 status is returned.  All following
       arguments to find are taken to be arguments to the command until
       an  argument  consisting of ‘;’ is encountered.  The string ‘{}’
       is replaced by the current file name being processed  everywhere
       it occurs in the arguments to the command, not just in arguments
       where it is alone, as in some versions of find.  Both  of  these
       constructions might need to be escaped (with a ‘\’) or quoted to
       protect them from expansion by the shell.  See the EXAMPLES sec-
       tion  for examples of the use of the ‘-exec’ option.  The speci-
       fied command is run once for each matched file.  The command  is
       executed  in  the  starting  directory.    There are unavoidable
       security problems surrounding  use  of  the  -exec  option;  you
       should use the -execdir option instead.

An example that recursively touches all *.log files from the current directory would be:

$ find . -name \*.log -exec touch {} \;
2 Comments :, more...

Extracting Audio/Video

by on Dec.01, 2007, under How To ..., Linux, Software

It’s really easy to extract either audio or video from a multimedia file using ‘ffmpeg‘. To extract audio only:


$ ffmpeg -i inputfile -vn -acodec copy outputfile

And for video only, replace ‘-vn‘ with ‘-an‘ and ‘-acodec‘ with ‘-vcodec‘.

ffmpeg is also commonly used as a transcoding tool.

Leave a Comment :, , more...

SSH Blacklisting

by on Nov.29, 2007, under How To ..., Linux, Software

After getting around 1500 failed ssh login attempts a day for a while on a server I manage, I decided to look into tools that automatically blacklist offending IPs.

Sshblack fits the bill perfectly. A HOWTO (including an init-script) for REDHAT-like systems is available from the OSS Watch Wiki.

1 Comment :, , , more...


Joining PDF Documents

by on Aug.03, 2007, under How To ..., Linux, Software

A quick search on the web reveals that the simplest (and most available) command to do so is:


$ gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -sOutputFile=finished.pdf file1.pdf file2.pdf

Source: Putting together PDF files by Scott Nesbitt on NewsForge

[Update: Feb 1, 2011] jpdftweak is probably a better option with many useful features.

Leave a Comment :, , more...

Software Keyboard and Mouse (KM) Switcher

by on May.15, 2007, under How To ..., Linux, Software

Stumbled across this great tool a few days ago:

Synergy

Description from the project homepage:

Synergy lets you easily share a single mouse and keyboard between multiple computers with different operating systems, each with its own display, without special hardware. It’s intended for users with multiple computers on their desk since each system uses its own monitor(s).

Redirecting the mouse and keyboard is as simple as moving the mouse off the edge of your screen. Synergy also merges the clipboards of all the systems into one, allowing cut-and-paste between systems. Furthermore, it synchronizes screen savers so they all start and stop together and, if screen locking is enabled, only one screen requires a password to unlock them all.

Leave a Comment :, more...

Mass conversion of images

by on May.07, 2007, under How To ..., LaTeX, Linux, Software

The following “one-liner” can be used to mass convert a given image format into another using the convert (part of ImageMagick) and basename tools:


$ for A in $(ls *.$SRC_TYPE); do convert $A $(basename $A .$SRC_TYPE).$DST_TYPE; done

where $SRC_TYPE is the file suffix of the original images (e.g. png) and $DST_TYPE is the file suffix of the type desired (e.g. eps).

Leave a Comment :, , , more...

Linux and DVD Regions

by on Apr.12, 2007, under How To ..., Linux, Software

Typically, Linux DVD playback software are capable of decrypting (libdvdcss must be installed) and playing back DVDs from any region, irrespective of the region code of the drive. Therefore, there should be no need to change the region code of the drive to watch discs from a different region. Regardless, there exists a very handy program that allows the user to change the region code and view other relevant information such as the number of changes remaining. It is called “regionset” and is available for Fedora from the Extras repository. The project website is:

http://linvdr.org/projects/regionset/

There is also a useful article on Linux.com about DVD playback:

DVD Playback HOWTO

Leave a Comment :, , more...

Gnome automount options

by on Mar.15, 2007, under How To ..., Linux, Software

On a default Fedora installation, Gnome will automatically mount (with some default options) any recognized volumes as soon as an external storage device is plugged in. While the defaults might be fine in general, sometimes it is necessary to supply some more options.

The tool responsible for automating the mount (gnome-volume-manager) gets additional options from the /system/storage/default_options/$fstype$/mount_options key of gconf — the Gnome “system registry”. One can check/modify existing entries (one a per user basis) by running:


$ gconf-editor /system/storage/default_options &

This should launch the graphical configuration editor with the default_options key selected. Under this key are the entries for different filesystems. Default mount options can then be changed by selecting the desired filesystem and editing the mount_options key.

If a required filesystem type is not listed, then it can be added by using the gconftool-2 utility. E.g. if we wanted to add the ext3 filesystem to the configuration database with the options “sync” and “uid=“, we would run the command:


$ gconftool-2 -t list –list-type string \
-s /system/storage/default_options/ext3/mount_options “[sync,uid=]”

Next time an external volume is plugged in, it will be mounted with the additional options specified! It should be noted that regardless of the options supplied via gconf, some mount options are always present: r(o|w),noexec,nosuid,nodev. At the moment I do not know how to change them.

The main motivation for finding out about the defaults is that I wanted to add “sync” as a default option. This causes data to be written immediately to the device, instead of being buffered first — a useful option to have for external devices. It should minimized data loss in case of an accidental removal (without running umount first). However, it should also be noted that for solid state drives (e.g. flash), this may result in a shortening of service life and poorer performance.

Update [Sun Mar 18, 2007]:
I’ve done some rudimentary throughput performance testing with both the sync and async modes and the performance hit with sync appears to be quite severe (at least 20 times slower than async) — even with a high performance HDD as the target. In light of this, I am removing sync from the list of default options. Given the type of data I’m likely to store on the device, speed is certainly more valuable than data integrity.

Test results:
Testing was carried out on a FAT32 volume. For each mode, both the transfer time and the subsequent un-mount time (indicating the time needed to flush the buffer) are shown.

async:

$ time dd if=/dev/zero of=/tmp/usb/dummy bs=8k count=130000 && time sudo umount /tmp/usb/
130000+0 records in
130000+0 records out
1064960000 bytes (1.1 GB) copied, 45.9035 seconds, 23.2 MB/s

real    0m46.026s
user    0m0.035s
sys     0m2.915s

real    0m11.680s
user    0m0.003s
sys     0m0.263s

sync:

$ time dd if=/dev/zero of=/tmp/usb/dummy bs=8k count=130000 && time sudo umount /tmp/usb/
130000+0 records in
130000+0 records out
1064960000 bytes (1.1 GB) copied, 969.384 seconds, 1.1 MB/s

real    16m13.525s
user    0m0.048s
sys     0m12.156s

real    0m0.842s
user    0m0.004s
sys     0m0.199s
1 Comment :, , more...

MATLAB – Interrupting function execution

by on Mar.02, 2007, under How To ..., Software

MATLAB itself does not provide any way of arbitrarily interrupting/pausing a function at mid-execution to examine the internal function stack/variables. It is a pity since such a mechanism would be an extremely valuable debugging tool.

Finally, I realized that this is not as impossible as it seems. In fact, it is EXTREMELY simple! All that is required are four lines of code in any script/function:

[statcode, result] = system('ls sometoken');
if statcode == 0
  keyboard
end

The system command is used to run system console commands. Here, a simple check is made for the existence of the file ‘sometoken’ in the current directory. If the file exists, statcode is set to zero and the if condition is satisfied. The keyboard command is then executed which causes MATLAB to pause execution and return control to the command line! In order to continue execution, ‘sometoken’ must be deleted/removed from the current directory and the command return must be issued.

The best place for this code is perhaps in any loop that might exist in the MATLAB script/function. Then, to cause execution to pause, one simply needs to create a file ‘sometoken’ in the current directory.

14 Comments :, more...