Linux System Notebook

Fix Ubuntu Swap Partition

2009-08-10T17:06:00.000Z

Find your swap partition:
sudo fdisk -l

Check the disk ID of the swap partition (e.g. /dev/sda8):
sudo vol_id /dev/sda8

Correct the UUID in the fstab file:
sudo nano /etc/fstab

"If Found" Disk Label

2009-08-10T14:59:00.010Z

We have so much portable data these days on usb drives, satnavs, cameras and mp3 players it's a nightmare and losing a device can mean losing irreplaceable photos and data files. Certainly where possible data should be encrypted and backedup locally or to online services such as Dropbox and Picasa but it isn't always possible or practical - such as on the holiday camera.

One simple approach that can at least identify you to anyone who finds a devices is "If Found" file.

At the very least create a simple text file with contact details and recovery information and save as "If Found.txt". It might help to suggest that you will cover all recovery costs and are interested more in the data card than recovery of the device.

However, to get more attention and especially instantly accessible from camera cards create an image:

Create a word processing file with the same contact information, perhaps even a photograph (might identify you in a crowded venue and helps humanise the situation), and formatted to a nice box shape inside a table, and export to PDF. The more the document looks like an ID card the more instantly identifiable as something interesting it will be.
Open the PDF in the Gimp, make sure the DPI on the import is good enough to produce a legible image, and use the autocrop tool to create the small box image.
Save as a JPEG "If Found.jpg" on the root directory of the media card or device.
Include both JPEG and TXT files on all MP3 players, camera data cards, satnavs, and USB disks, etc. On a Garmin savnav you can set an image as the splash screen. This is displayed before the password request. Placing the image in the root of internal memory ensures it is also the first image in the picture viewer tool. On a regular digital camera place the image on the root of the disk to keep a position of image number 1. Place a copy on the internal memory too if available (for my Kodak this meant actually photographing the image from the screen to the internal memory).

PC Security

2009-08-07T11:32:00.002Z

As I'm often asked for advice on PC security for Windows machines, here is a brief summary of the free and opensource tools available for Windows users.

Windows
Windows itself has its own important defences. Make sure that your auto-update is turned on. Out-of-date system software is going to be more vulnerable to attack.

Windows Defender http://www.microsoft.com/windows/products/winfamily/defender/default.mspx

"Windows Defender is software that helps protect your computer against pop-ups, slow performance, and security threats caused by spyware and other unwanted software by detecting and removing known spyware from your computer. Windows Defender features Real-Time Protection, a monitoring system that recommends actions against spyware when it's detected, minimizes interruptions, and helps you stay productive."

Anti-Virus Software
There are a number of basic free anti-virus software options.
AVG http://free.avg.com/

"Antivirus and antispyware protection for Windows available to download for free. LinkScanner Active Surf-Shield checks web pages for threats at the only time that matters – when you’re about to click that link. AVG Anti-Virus Free Edition. Award-winning antivirus and antispyware. Real-time safe internet surfing and searching. Quality proven by 80 million of users. Easy to download, install and use. Protection against viruses and spyware. Compatible with Windows 7, Windows Vista and Windows XP. AVG Anti-Virus Free Edition download and installation is only available for single computer use for home and non commercial use."

ClamAV http://www.clamwin.com

"ClamWin is a Free Antivirus program for Microsoft Windows 98/Me/2000/XP/2003 and Vista. ClamWin Free Antivirus comes with an easy installer and open source code. You may download and use it absolutely free of charge. It features: High detection rates for viruses and spyware; Scanning Scheduler; Automatic downloads of regularly updated Virus Database. Standalone virus scanner and right-click menu integration to Microsoft Windows Explorer; Addin to Microsoft Outlook to remove virus-infected attachments automatically."

Confickr Eyechart http://www.confickerworkinggroup.org/infection_test/cfeyechart.html

"Conficker (aka Downadup, Kido) is known to block access to over 100 anti-virus and security websites. If you are blocked from loading the remote images in the first row of the top table above (AV/security sites) but not blocked from loading the remote images in the second row (websites of alternative operating systems) then your Windows PC may be infected by Conficker (or some other malicious software). If you can see all six images in both rows of the top table, you are either not infected by Conficker, or you may be using a proxy server, in which case you will not be able to use this test to make an accurate determination, since Conficker will be unable to block you from viewing the AV/security sites."

Three Broadband Router Logs

2009-07-19T16:41:00.008Z

This script automatically updates a local /var/log/ppp.log log from the latest router logs (otherwise the router logs are not timestamped and are deleted on router reboot).

Data from /var/log/ppp.log can be used track and audit broadband usage and to effectively compare with provider billing.

#!/bin/bash
# Retrieve router broadband usage logs.
# See /var/log/ppp.log for results.
# Run automatically ever hour by adding to /etc/cron.hourly as routerlog.
# Unfortunately as the router does not timestamp its entries - exactly duplicate entries will not be added to the log (how likely is this?)
# New entries to the log are sorted alphabetically (necessary for comm) rather than chronologically. It does not effect the overall audit.

user=admin
password=admin
log="/var/log/ppp.log"
search="es\." # happily identifies both lines for connection time and bytes sent and received
router="192.168.1.1"
sleeptime=4 # telnet pause
temp="/tmp/ppp.log"

# get latest router log

(sleep $sleeptime; echo "admin"; sleep $sleeptime; echo "admin"; sleep $sleeptime; echo "cat $log | grep '$search' "; sleep $sleeptime; echo "quit") | telnet $router | grep "$search" | tac | tr -d '\r\n' | sed 's/minutes\./\minutes\.\n/g' | tac | sed 's/Connect/ Connect/g' | sort > "$temp"

# compare overlapping lines in new log entries and complete log

touch "$log"
lines="$(wc -l $temp | cut -f 1 --delimit=" ")"
cat "$log" | grep "$search" | tail -$lines | sort -u > "$temp.0"
comm "$temp" "$temp.0" -2 -3 | cut -f 1- --delimit=" " > "$temp.1"
new="$(wc -l $temp.1 | cut -f 1 --delimit=" ")"
if [ $new -gt 0 ]
then
date +"(%Y/%m/%d %H:%M)" >> $log
cat "$temp.1" >> "$log"
fi
rm "$temp" "$temp.0" "$temp.1"

Log Broadband Use (Three.ie)

2009-07-18T19:15:00.004Z

Mobile broadband is often the best we have and is mostly reliable and fast enough - however over quota charges are horrendous. It is important to keep track of usage and remaining data allowance. There is no automatic way to do this on Linux, especially on a network used by multiple machines.

Three provides a web interface to latest data but it isn't always convenient and doesn't record a usage history - essential for tracking down extraordinary usage problems.

This simple script presents a solution, accessing the data from my.three.ie - storing the quota remaining in /var/log/threebroadband.log. Add it to /etc/cron.hourly as "threebroadband". I like to use Wanda-the-Fish on the Gnome panel to be able to quickly check on usage (add the custom command "tac /var/log/threebroadband.log" to the preferences instead of "fortune").

#!/bin/bash
# Check Three.ie broadband limit remaining.
# See /var/log/threebroadband.log for results (or add Wanda-the-Fish to the Gnome panel with the custom command "tac /var/log/threebroadband.log"
# Run automatically ever hour by adding to /etc/cron.hourly as threebroadband

# Login to my.three.ie

curl -s -c /tmp/cookie https://my3.three.ie/myaccount/selfcare.do > /dev/null
curl -s -b /tmp/cookie -c /tmp/cookie "$(curl -s -b /tmp/cookie -c /tmp/cookie https://my3.three.ie/mylogin//login?service=https%3A%2F%2Fmy3.three.ie%2Fmyaccount%2Fselfcare.do | grep Click | cut -f 2 --delimit="\"")" > /dev/null

# Extract limit to logs

echo $(date +"%y/%m/%d %H:%M") MB: $(curl -s -b /tmp/cookie -c /tmp/cookie https://my3.three.ie/myaccount/selfcare.do | grep "Mobile" -A "14" | tail -1 | cut -c 7-) >> /var/log/threebroadband.log

rm /tmp/cookie

BBC iPlayer Download

2009-07-13T13:15:00.001Z

You can download the BBC iPlayer stream with mplayer (thank you to B. Halpin for the tip) with, for example:

wget http://www.bbc.co.uk/iplayer/aod/playlists/zy/7p/k0/0b/RadioBridge_intl_2000_bbc_radio_fourfm.ram

mplayer -dumpstream -bandwidth 9999999 `cat RadioBridge_intl_2000_bbc_radio_fourfm.ram`

The URL for the stream is available in the iPlayer page source code (search the text for ".ram").

Proxy Server

2009-07-01T13:32:00.004Z

Even at home a proxy server can be useful and is easy enough to setup: it helps keep track of downloads easily, can be used to limit access to content, and helps minimise download expenses across multiple machines for updates and other downloads.

The proxy server itself is squid. A useful reporting program is sarg.

sudo apt-get install squid sarg

Squid

You will need to edit the /etc/squid/squid.conf file (as superuser) to uncomment, and edit if necessary, down in the custom settings section the lines which read something like:

sudo nano /etc/squid/squid.conf

acl our_networks src 192.168.1.0/24
http_access allow our_networks

(and in my case the visiblehostname (visible_hostname laptop))

Sarg

Sarg is mostly automatic, however you may have to edit /etc/squid/sarg.conf to customise and correct some errors (spelling of "monthly").

sudo nano /etc/squid/sarg-reports.conf

DAILY=Daily
WEEKLY=Weekly
MONTHLY=Monthly

Log rotation for the squid files need to be changed to monthly:

sudo nano /etc/logrotate.d/squid

Reports will be found in /var/www/squid-reports/index.html

With a web server such as Apache installed: reports are available through a browser at: http://server.local/squid-reports or whatever name of the machine is instead of "server.local".

Set Proxy

To apply system-wide proxy edit .bashrc to add an export line:

nano .bashrc

export http_proxy="http://server.local:3128"

Linux Desktop Breaking News

2009-06-18T10:28:00.004Z

Despite being left behind by the BBC's breaking news software, the Linux desktop is back in action with news headlines. Gwibber is a blogging/twitter client that is beautifully presented and works nicely and well integrated with the Gnome desktop.

sudo apt-get install gwibber

Apart from adding your friends on Facebook and Twitter, add the news services that are popping up now from RTENews, RTETransport, BBCNews, BBCTech, and Slashdot, etc. Perfect solution.

Copy Ubuntu LiveUSB

2009-06-12T09:15:00.003Z

Ubuntu comes with a very useful tool "USB Startup Disk Creator" for creating a LiveUSB (similar to the LiveCD, but with persistence and is more easily customisable). However strangely enough once booted from the LiveUSB you cannot then use the tool to create another or to copy a version of the Ubuntu installation media - even though the complete CD is available on the LiveUSB.

The solution is a single command to recreate the Ubuntu CD from the LiveCD's mounted /cdrom directory. This can then be burned to physical CDs or used by the "USB Startup Disk Creator" tool to create another USB. It might also be booted by Qemu or Vmware as a virtual system.

genisoimage -D -r -V "Ubuntu 9.04 i386" -cache-inodes -J -l -b syslinux/isolinux.bin -c syslinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o cd.iso -m "casper-rw" /cdrom/

See https://help.ubuntu.com/community/LiveCDCustomization for further customisation tips

Create Disk Image

2009-06-12T08:10:00.004Z

Copy Physical Disk

To create a disk image from an actual disk is relatively straightforward - a simple dd command from the physical disk to file. For example from /dev/sda to harddisk.img:

sudo dd if=/dev/sda of=harddisk.img

(Warning: do not mix these up. The reverse is seriously damaging.)

Make New Harddisk Image

However, to create a complete disk image from scratch is less trivial; but of course possible. The following script demonstrates the series of commands which is no more than a batch of simple commands with the added ability to take an argument for the size in MB of the size of disk required (e.g. to create a 100MB disk: ./makediskimage.sh 100). It creates a single bootable FAT32 partition with syslinux. Other configurations are equally possible.

# makediskimage.sh

sizeMB=$1
size=$(echo $(($sizeMB*1024*1024/512)))
# set size of disk
dd if=/dev/zero of=harddisk.img bs=512 count=$size
# equivalent to: qemu-img create -f raw harddisk.img 100M
parted harddisk.img mktable msdos
# create partition table
parted harddisk.img "mkpart p fat32 1 -0"
# make primary partition, type fat32 from 1 to end
parted harddisk.img mkfs y 1 fat32
# make fat32 filesystem on partition 1, without confirmation
parted harddisk.img toggle 1 boot
# make partition 1 bootable
parted harddisk.img unit b print
# list partition table (in bytes)
offset=$(parted harddisk.img unit b print | tail -2 | head -1 | cut -f 1 --delimit="B" | cut -c 9-)
# get offset
# sometimes 512, 16384 or 35226 (512 bytes per unit by 63 cylinders)
sudo syslinux -o $offset harddisk.img
# add boot code to partition 1
dd if=/usr/lib/syslinux/mbr.bin of=harddisk.img conv=notrunc
# copy master boot record to disk

Mount Disk Image

To mount a partition image is straightforward, for example:

sudo mount partition.img /mnt/diskimage

To mount a partition on a complete disk image it only requires an offset to the partition to mount (here on a temporary mount point):

mkdir /tmp/harddisk.img
sudo mount harddisk.img /tmp/harddisk.img -o loop,offset=16384 # 512*32 cylinders

QEMU

Qemu is the opensource virtualisation software - capable of boot raw disk images and others. To install, include graphical launcher (useful for learning commands):

sudo apt-get install qemu qemu-launcher

To boot harddisk.img with Qemu:

/usr/bin/qemu -boot c -m 512 -hda 'harddisk.img' -net nic,vlan=0 -net user,vlan=0 -localtime

LIVE CD/USB

To add the option of booting into a harddisk image to a live CD/USB boot, for example Clonezilla on an image called clone.img, add to the following lines to syslinux.cfg:

label clonezilla
menu label ^Clonezilla
kernel memdisk
append initrd=/clone.img

(Note: Clonezilla at present needs it's own syslinux and mbr.bin as provided in the utils directory in order to boot properly)

For Syslinux boot disk images can also be compressed with gz to save space:
gzip harddisk.img
mv harddisk.img.gz harddisk.imz

Change the line in syslinux.cfg to:
append initrd=/clone.imz

YouTube Video Download

2009-06-10T18:27:00.002Z

Nice simple program to download YouTube videos and avoid interruption and the download costs of repeated downloads online.

To install:
sudo apt-get install youtube-dl

To use:
youtube-dl http://www.youtube.com/watch?v=Vrhgk8Fa_QE

Recompile Modules

2009-06-09T10:33:00.003Z

A recent problem with the Alsa sound driver (after an aborted attempt to install the linuxant modem driver) seemed intractable. Remove and install did nothing. A complete reinstall seemed the only likely solution. Module-Assistant to the rescue helping me recompile the Alsa module and get sound again after a restart:

to install:
sudo apt-get install module-assistant
to run:
sudo m-a

Inside the Virtual Anthill

2009-06-09T08:47:00.003Z

"Inside the Virtual Anthill: Open Source Means Business"
BBC Radio 4, Broadcast June 1, 2009

"Gerry Northam goes behind the scenes to investigate 'open source' computer software. Much has been said about the likes of free web browser Firefox and the operating system Linux, but little about how thousands of programmers scattered around the world collaborate in a 'virtual anthill' to create products that rival more commercial offerings. Gerry finds out how it is done and shows how its ethos is being applied to other kinds of business, with some startling results."

http://www.bbc.co.uk/programmes/b00kp806

Good programme to introduce the notion of open source with examples from Mozilla, Linux, IBM and others. A good resource for educators and community activists. Available on iplayer. No downloadable podcast yet, however you can download the stream with mplayer (thank you to B. Halpin for the tip) with:

wget http://www.bbc.co.uk/iplayer/aod/playlists/zy/7p/k0/0b/RadioBridge_intl_2000_bbc_radio_fourfm.ram

mplayer -dumpstream -bandwidth 9999999 `cat RadioBridge_intl_2000_bbc_

radio_fourfm.ram`

The URL for the stream is available in the iplayer page source code.

Bash Script Debugging

2009-05-12T22:59:00.001Z

A seemingly very simple technique which is a lifesaver in checking bash scripts: instead of running the script, e.g.

./renamefiles.sh

run

bash -x ./renamefiles.sh

for a command by command report of activity in the script. Each command is preceded by a "+". Lines without a "+" are ordinary script outputs.

PPM Compression

2009-05-10T17:13:00.008Z

For compression of archives and transport of large files, it is worth trying PPM "Prediction by Partial Matching" (http://en.wikipedia.org/wiki/Prediction_by_Partial_Matching).

To install in Linux (though the author does not recommend this application for general use):
sudo apt-get install ppmd

For help:
man ppmd

To compress the file archive.tar (creates archive.pmd):
ppmd e archive.tar

Incredibly slow but can be twice as compressing as zipping. A good trade-off for longterm backups. However in simple tests on binary files it did not perform as well as bzip2 or lzma (http://en.wikipedia.org/wiki/LZMA) which gave best results for general files. The author does explain in the man file that his program is optimised for natural language rather than "noisey" files.

Recover Vista, and Edit Registry from Linux

2009-05-06T01:59:00.004Z

Problems occur with Vista if partition or disks get changed around.

I moved a partition slightly using Linux gparted. This was successful but the problem was when Windows boot breaks as it can't find its old partition ID.

So, if possible, boot with Vista recovery (pressing the F8 key immediately on Vista boot) and try the automatic recovery. Another option is the Vista installation DVD; or failing that download and burn the Vista Recovery DVD from http://neosmart.net/blog/2008/windows-vista-recovery-disc-download/.

However, this might not be the end of the problem. Dell machines have a separate boot and recovery partition before the system partition. Problems arise when the recovery procedure allocates the drive names in disk order: C for RECOVERY and D for OS. On reboot Vista then gets very confused looking for files in C: and insisting on keeping them in D:.

The solution, hopefully, is to run the command prompt from the recovery boot again. Windows boot files are now longer directly editable for some silly reason, so you'll need to use the bcdedit program which will be available from the recovery boot (EasyBCD is a fine program for editing the boot record but unfortunately this isn't going to be available on a disabled system).

bcdedit
lists the current boot record.

bcdedit /v
lists the current boot record with full device IDs.

In order to change device names, using the device IDs as listed, use the "/set" switch followed by the ID device and the full correct partition. For example:

bcdedit /set {549a88e6-38fe-11de-8cb6-d0e4c85376d6} device partition=c:
bcdedit /set {549a88e6-38fe-11de-8cb6-d0e4c85376d6} osdevice partition=c:

Try a reboot.

Believe it or not, problems can persist (such a bloody annoying system). Although the BCD records lists the system device as C: again, Vista decides to suddenly use E:. The problem this time lies in the registry. Somewhere along the line device C: is listed as a mounted device and unavailable. And unfortunately it becomes impossible to edit the registry from a disabled system.

However, thankfully there is a Linux solution to editing the registry offline.

From a live CD (download from http://www.ubuntu.com/getubuntu/download), USB system or second installed system, install chntpw ("Change NT Password"). Apart from its obviously main purpose of changing passwords this neat program also allows simple but reliable offline registry editing.

To install on debian-based Linux system:
sudo apt-get install chntpw

To learn more about the program:
man chntpw

To run on, for example a Vista disk mounted at /media/OS, run with the "-e" edit switch as:
chntpw -e /media/OS/Windows/System32/config/SYSTEM

Once inside the program, then:

For help, use: ?
To list registry keys: ls
To list the key : ls MountedDevices
To delete all : rdel MountedDevices
And, to quit: q

Reboot, pray.

Ubuntu Screensaver Options

2009-05-04T23:01:00.003Z

Options seem to have disappeared from the Ubuntu screensavers. Install xscreensaver and xscreensaver-data-extras to get them back.

Customise what text is displayed in text based screensavers, for example:

Time: add date +"%H:%M %d/%m/%y" to the program option.
News: add, for example, http://www.rte.ie/rss/news.xml to the feed option.

Javascript Redirect

2009-05-04T15:53:00.005Z

Redirect to another domain:

<script type="text/javascript">

</script>

Upgrade Server from CD

2009-05-04T15:34:00.004Z

To bandwidth add the Ubuntu Server CD (http://www.ubuntu.com/getubuntu/download-server) to the server before upgrading.

sudo apt-cdrom add

then

sudo apt-get update
sudp apt-get upgrade
sudo apt-get dist-upgrade

Moodle on a VMWare Guest

2009-05-03T12:48:00.003Z

If you experience problem with the hostname (though if your system was set up correctly with DHCP the hostname should resolve through the VMWare NAT network and nameservers) :

Change line in /usr/shared/moodle/config.php
$CFG->wwwroot = 'http://'.$_SERVER["SERVER_NAME"].'/moodle';

Otherwise you might also have to change from, for example, ubuntu.localdomain to ubuntu.local

Rename Files

2009-05-03T11:23:00.002Z

In this example it removes the extension ".auto" (the "\." is required in the find expression).

find -name "*.auto" -print0 | xargs -0 rename 's/\.auto//'

Deleting Emails

2009-04-26T17:01:00.002Z

I regularly need to delete emails without reading them from a forwarding address at my Vodafone phone. In an adaptation of another script:

#!/bin/bash

# Delete POP emails
# Adapted from http://www.cyberciti.biz/tips/remove-or-delete-all-emails-message-from-a-pop3-server.html

username="myemailaddress"
password="somethingsecret"
server="pop.vodafone.ie"
MAX_MESS=100

delete(){
sleep 2
echo USER $username
sleep 1
echo PASS $password
sleep 2
for ((j=1;j<=$MAX_MESS;j+=1))
do
echo DELE $j
sleep 1
done
echo QUIT
}

delete | telnet $server 110

Fix Ubuntu Splash Screen

2009-04-07T08:21:00.002Z

Find your swap partition:
sudo fdisk -l

Check the disk ID of the swap partition:
sudo vol_id /dev/sda8

Correct the UUID in the resume file:
sudo nano /etc/initramfs-tools/conf.d/resume

The Apostrophe Problem

2009-04-05T10:10:00.003Z

In Ireland especially we have the terrible apostrophe problem in names and then problems with SQL commands having quotes breaking up and causing errors. Damn, it's a simple answer: replace the single quote with two single quotes before further commands.

personwithapostrophe=Replace(person,"'","''")

Multiple Access Reports

2009-04-04T12:14:00.014Z

There's the odd problem of generating multiple reports from Access - such as individual invoices, individual rosters, etc. A single report as a pdf is not easily emailable or easily chopped up for each individual. The solution:

To create individual roster reports to be printed to pdf (see pdfcreator) based on the report timetable_roster: create a new blank report called, for example here, print_roster.

In the code of print_roster include, for example (SQL variations in bold):

Private Sub Report_Open(Cancel As Integer)

' create roster reports

Dim rst As DAO.Recordset

Dim db As DAO.Database

Dim hold As String

Set db = CurrentDb

Set rst = db.OpenRecordset("SELECT roster.person FROM Roster GROUP BY roster.person;;")

If rst.BOF Or rst.EOF = True Then GoTo jump_out

rst.MoveFirst

Do While Not rst.EOF

person = rst!person

personwithapostrope=Replace(person,"'","''")

DoCmd.OpenReport "timetable_roster", acViewPreview, , "person='" & personwithapostrophe & "'", , person

DoCmd.OpenReport "timetable_roster", acViewNormal, , "person='" & personwithapostrophe & "'", , person

DoCmd.Close acReport, "timetable_roster"

rst.MoveNext

Loop

jump_out:

rst.Close

Set rst = Nothing

End Sub

Private Sub Report_Page()

DoCmd.Close acDefault, , acSaveNo

End Sub

In the code of the original timetable_roster report, include:

Private Sub Report_Open(Cancel As Integer)

' change report title

If Not IsNull(Me.OpenArgs) Then

Me.Caption = OpenArgs

End If

End Sub

When is Easter?

2009-04-04T12:12:00.000Z

ncal -e 2009

Using BloGTK with Blogger

2009-03-15T13:05:00.001Z

Talk about weird piece of information that is difficult to find: set server to http://www.blogger.com/api/RPC2.

Change Dates

2009-03-15T12:35:00.002Z

I had a problem recently with a camera that has serious battery problems and never has the correct date. It screws up the filing of photos later.

As usual, there was a simple solution: the linux command touch - which I only ever used for creating empty files.

To change the dates on all files in a folder to the 17th of March 2009 at teatime it is as simple as:

touch * -t "200903171800"

Online Backup and Synchronising

2009-03-07T15:21:00.003Z

An excellent tool I can across recently is Dropbox - which works on both Windows and Linux and allows you to instantly backup files online and synchronise with more than one machine or device. You can set some folders to share with other people for collaborative work. Works excellently and even allow simple web access to your files while on cybercafe or other machines.

Huawei E220 USB Modem on Ubuntu

2008-09-21T14:29:00.002Z

The Huawei E220 is installed on a USB connection. It provides an always-on broadband connection on a 3G network or falls back to dialup speeds on the lesser GPRS (2.5G) network.

No support is generally offered for Linux, but it does work on Linux.

Ubuntu Hardy

Things are getting so much easier. Today on Ubuntu Hardy it just works. The modem is detected automatically as /dev/ttyUSB0.

Configuration is straightforward via the network icon on the panel or via System-Administration-Network on the Gnome menu.

Edit the point-to-point connection to enable the connection as GPRS with the appropriate ISP access point (e.g. for 3 Ireland, it is 3ireland.ie), or serial modem with the phone number *99# may also work.
Set the modem port to /dev/ttyUSB0.
Switch the default route, ISP nameservers, and retry on in the options tab.

The lights on the modem itself will let you know what is happening and if things are working.

Recovering Partitions

2008-06-09T10:11:00.007Z

Partitioning can go wrong. Yes, we should have backups. But there are times when you or some utility screws up the partition table.

Ubuntu

I've found reconstructing the partition table straightforward enough with an essential tool: the Ubuntu Live CD.

By booting from the CD to provide good clean access to the harddisk, even when your system partition is still operational.

Once in Ubuntu, open a terminal (from accessories, or Ctrl+Alt-F1).

fdisk

For a simple check on the partition table and disk:

sudo fdisk -l

It is a straightforward way of checking the device name of the disk in question, however although fdisk is a useful partition tool, I advise using parted in this case for its rescue command.

Now before you do anything: it is important to state that everyone's problem is different and this is not a solution to your exact problem. Please make a complete backup of all intact partitions; or make a complete disk image; and at the very least make a backup of your partition table to a usb drive or something (click for details). Editing partitions is a non-trivial matter and although strictly speaking you are only editing the partition index and the actual data on the disk always remains untouched you could lose track of valuable partition data, cause hours of heartache, and by rebooting into a badly reconstructed partition table perhaps cause data damage. Backup.

parted

To run parted (on, for example, /dev/sda):

sudo parted /dev/sda

To get a list of the current partition table:

print

To change the units to a more precise cylinder count, or sector count, use:

unit cyl

or

unit s

Where there are incorrect partition entries overlapping the locations of the partition to be recovered (you are 100% sure!), use the remove command, e.g.

rm 3

Then rescue partitions, use the rescue command with the start and end cylinders (or sectors) estimated for the partition. This can be tricky and it can seem to miss where the start or end values are only slightly off; but other times it will cycle through a list of possible partitions to be recovered based on the filesystem headers that it finds (the first possible partition might not always be the one you are looking for, check the size, as old scraps of filesystems may be lying around and bits of partitions that have been moved will also exist close to the partition we are looking for).

rescue start end

It all works best if you are reconstructing a partition table, or section of, that started at the beginning and continues partition after partition without significant gaps. It is fairly straightforward then to begin a search, recover the first partition, then continue the search at the next available cylinder. Otherwise, a disk utility like Autopsy can help in viewing the disk and search for keywords such as, for example with an ntfs partition "Press Ctrl+Alt+Del to restart", to find the start of the deleted partition.

fsck

Recovered partitions can be "tested" be using the regular Ubuntu desktop to mount and explore the files, though the command fsck will provide a more thorough check.

Regular partition editing can be done a lot more easily from the main menu in System/Administration.

Do as I Do

2008-06-07T11:15:00.003Z

Gnome-Do is the biz. Run command, programs and searches easily with a strike of the "super"-but-looks-like-the-"Windows"-key and the spacebar.

https://wiki.ubuntu.com/GnomeDo

Adding it to the sessions startup programs saves another step.

gnome-do -q

IBM Lotus Symphony

2008-06-06T11:29:00.004Z

In world where we already have opensource OpenOffice.org it probably isn't that important but IBM Lotus Symphony is a available free (based on an older version OpenOffice.org and taking advantage of its dual license to not release it as opensource) to run on Linux and Windows.

But interesting to look at and yet another tool for free software in the office.

Installs perfectly well on Ubuntu, needs a fair bit of space, but straightforward. However a fix to get it to run is required at present (since it is installed as root permissions are a bit screwed).

sudo chown -R username ~/.lotus/

http://symphony.lotus.com

Printer Troubles

2008-06-05T16:59:00.004Z

I have a printer setup which gets frustrating. There is no telling if my old HP4L (now with a USB-to-parallel adapter cable) will be recognised by Ubuntu and at which USB port. I can detect no pattern to the misbehaviour or to how to get it to be recognised again beyond deleting the printer, restarting, and reintroducing the USB cable to a naive Ubuntu.

The printer is found as "unknown".

To rename it takes:

and a restart of cups as:

I also discovered simple cups administration at:

Maybe there's a better way of training my printer?

Live Desktop

2008-06-05T15:59:00.010Z

Further to the previous post which used ImageMagick to add live weather views to the desktop, this post extends the idea with a selection of webcam images, a local newspaper front page, and weather radar to create a real desktop window on the world.

#!/bin/bash

wallpapers="/usr/share/backgrounds/"
wallpaper="simple-ubuntu.png"
weather="http://www.met.ie/weathermaps/latest_radar.gif"
cam1="http://www.galway.net/galwayguide/webcam/cam05/current.jpg"
cam2="http://www.dublintraffic.com/Site0Camera10.jpg"
cam3="http://www.dublintraffic.com/Site0Camera21.jpg"
cam4="http://www.dublintraffic.com/Site0Camera44.jpg"
cam5="http://www.dublintraffic.com/Site0Camera79.jpg"
cam6="http://www.dublintraffic.com/Site0Camera41.jpg"
cam7="http://www.dublintraffic.com/Site0Camera47.jpg"

convert -page +0+0 $wallpapers$wallpaper -page +400+400 $weather -page +800+300 $cam1 -page +700+600 $cam2 -page +800+900 $cam3 -page +1400+300 $cam4 -page +1500+900 $cam5 -page +1200+700 $cam6 -page +1600+600 $cam7 -flatten $wallpapers/radar.jpg

Include more of that ImageMagick magic and you've got a real fun addition to your desktop world - something not possible under Windows Vista despite all its boasting.

radar

#!/bin/bash
# radar
# live desktop script

wallpapers="/usr/share/backgrounds/"
wallpaper="simple-ubuntu.png"
img1="http://www.galway.net/galwayguide/webcam/cam05/current.jpg"
cap1="1. Galway Harbour"
pos1="+800+300"
img2="http://www.dublintraffic.com/Site0Camera10.jpg"
cap2="2. Wolfe Tone Quay"
pos2="+700+600"
img3="http://www.dublintraffic.com/Site0Camera41.jpg"
cap3="3. Wellington Quay"
pos3="+800+900"
img4="http://www.dublintraffic.com/Site0Camera44.jpg"
cap4="4. Dame Street, Dublin"
img5="http://www.dublintraffic.com/Site0Camera79.jpg"
pos4="+1400+300"
cap5="5. Mad Cow, M50, Dublin"
pos5="+1500+900"
img6="http://www.dublintraffic.com/Site0Camera3.jpg"
cap6="6. O'Connell Street"
pos6="+1200+700"
img7="http://www.dublintraffic.com/Site0Camera21.jpg"
cap7="7. Dame Street, Dublin"
pos7="+1600+600"
img8="http://www.dublintraffic.com/Site0Camera85.jpg"
cap8="8. Naas Road, Dublin"
pos8="+350+800"
img9="http://www.met.ie/weathermaps/latest_radar.gif"
cap9="9. Weather Radar"
pos9="+400+400"
img10="http://www.met.ie/weathermaps/nat00_weather.png"
cap10="10. Weather"
pos10="+200+300"
img11="http://www.eveningecho.ie/pdf/front.pdf"
pos11="+50+800"

convert -quiet -caption "$cap1, %c %f" $img1 -background black +polaroid /tmp/img1.tif
convert -quiet -caption "$cap2, %c %f" $img2 -background black +polaroid /tmp/img2.tif
convert -quiet -caption "$cap3, %c %f" $img3 -background black +polaroid /tmp/img3.tif
convert -quiet -caption "$cap4, %c %f" $img4 -background black +polaroid /tmp/img4.tif
convert -quiet -caption "$cap5, %c %f" $img5 -background black +polaroid /tmp/img5.tif
convert -quiet -caption "$cap6, %c %f" $img6 -background black +polaroid /tmp/img6.tif
convert -quiet -caption "$cap7, %c %f" $img7 -background black +polaroid /tmp/img7.tif
convert -quiet -caption "$cap8, %c %f" $img8 -background black +polaroid /tmp/img8.tif
convert -quiet -caption "$cap9, %c %f" $img9 -background black +polaroid /tmp/img9.tif
convert -quiet -caption "$cap10, %c %f" $img10 -background black +polaroid /tmp/img10.tif

convert -quiet -scale 50% $img11 /tmp/img11.gif
convert -quiet /tmp/img11.gif -background black +polaroid /tmp/img11.tif

convert -quiet -page $pos11 /tmp/img11.tif -page $pos10 /tmp/img10.tif -page $pos8 /tmp/img8.tif -page $pos7 /tmp/img7.tif -page $pos6 /tmp/img6.tif -page $pos5 /tmp/img5.tif -page $pos4 /tmp/img4.tif -page $pos3 /tmp/img3.tif -page $pos2 /tmp/img2.tif -page $pos1 /tmp/img1.tif -page $pos9 /tmp/img9.tif -page +0+0 $wallpapers$wallpaper -reverse -flatten $wallpapers/radar.jpg

Desktop Weather

2008-05-23T10:47:00.020Z

Add a simple script to /etc/cron.hourly and you have an updated weather map (in this case the Irish rain radar map) which you can stretch to any size you need on the desktop.

#!/bin/bash
wget -qmnd http://www.met.ie/weathermaps/latest_radar.gif -O /home/ian/Desktop/weather.gif

Or even better, if you've installed imagemagick, combine the latest radar with your wallpaper automatically. Set your wallpaper to radar.jpg.

#!/bin/bash
wallpapers="/usr/share/backgrounds/"
wallpaper="simple-ubuntu.png"
weather="http://www.met.ie/weathermaps/latest_radar.gif"

convert -page +0+0 $wallpapers$wallpaper -page +400+400 $weather -flatten $wallpapers/radar.jpg

Modem Download Volume Logging

2008-05-22T17:49:00.008Z

There doesn't seem to be an straightforward way to keep track of volume downloads and uploads over a modem on Ubuntu or any Linux - especially using NetworkManager.

Volume is important these days when on a broadband package with download limits; helps too when checking the bill; and helps to calm the nerves and perhaps use more of your limit than otherwise.

Based on a discussion on the forums (http://ubuntuforums.org/showthread.php?t=381138) I have mostly solved the problem with a simple script and the help of Wanda the Fish.

A simple script, called "logup" is placed in "/etc/network/if-up.d". It runs on interface up and adds to the /var/log/volume.log.tmp every 5 minutes (making sure that the logs are nearly accurate even on a system crash or power down without logging off first). A second script called "logdown" is placed in "/etc/network/if-down.d". It completes the current session log if the interface is shut down calmly.

On the Gnome desktop I've added Wanda the fish and changed her command to "tac /var/log/volume.log" (tac is cat in reverse, by the way).

/etc/network/if-up.d/logup

#!/bin/bash

interface="ppp0"
log="/var/log/volume.log"
tmp="$log.tmp"

echo >> $log
date >> $log
tail -1 $tmp >> $log
rm $tmp

total=0
for part in `cat $log | grep $interface`
do let "total+=part"
done
let "total/=(1024*1024)"
echo "Total: $total MB" >> $log

while true; do cat /proc/net/dev | grep $interface | tr ":" " " | tr -s " " |cut -d' ' -f2,3,11 >> $tmp; sleep 5m; done &

/etc/network/if-down.d/logdown

#!/bin/bash

interface="ppp0"
log="/var/log/volume.log"
tmp="$log.tmp"

cat /proc/net/dev | grep $interface | tr ":" " " | tr -s " " |cut -d' ' -f2,3,11 >> $tmp

Organise Files by Size

2008-04-26T11:41:00.008Z

I had the problem of about a million recovered files, some scraps, in a single folder. File management doesn't scale well to a million files; so this script reorganises the contents by file size. In this case it uses a scale from 10Mb to 0 in 100k increments.

organisebysize.sh

for i in `seq 10000 -100 0`
do
mkdir "$i"k
mv -v $(find -maxdepth 1 -type f -size +"$i"k) "$i"k
rmdir "$i"k
done

Alternate

Of course even this had scale problems for so many files. My final fix was a command to generate a batch file.

find -type f -printf "mkdir %bblocks; mv %p %bblocks \n" > ../movefiles.sh

Recovering Photos

2008-04-26T11:36:00.003Z

Regardless of operating system or file structure a great last resort for recovering photos and other jpeg images is jpegextractor.

http://schmidt.devlib.org/software/jpeg-extractor.html

In my case, treating the whole disk as a single file and ignoring any file structure which had been destroyed, the command was:

java jpegextractor /dev/sdb1

to attempt a recovery of files from a harddisk on a usb connection.

Search Folders

2008-04-13T17:42:00.002Z

Update to Nautilus script to automatically fill folders on a folder name

autofill

# AutoFill, version 1.2, by Ian Spillane (ispillane at ccoc dot ie), 2008.
# A Nautilus script to automatically populate a folder with links to files matching the folder name. A simple but effective implementation for virutal folders for music, images, or any file type.
# Released on the GPL (http://www.gnu.org/copyleft/gpl.html).

# To do: Implementation of further search options, multiple key words, file types, etc., still based on a simple syntax in the folder name, e.g. black|white[image].

search=$*

if test -z "$*"
then
# If no arguments: install as Nautilus script
cp autofill ~/.gnome2/nautilus-scripts/autofill
echo "Installed as Nautilus script. To use, right-click on a folder in Nautilus and select scripts."
else
cd $search
# Make links in folder for every file matching name, except for .Trash
for f in $(find ~ | grep -i "$search" | grep -v "$(pwd)" | grep -v ".Trash" | sed s/[\ ]/{}/g)
do
file=$(echo $f | sed s/{}/\ /g ) # return spaces
link="$(date +"%H%M%S-%N")-$(basename "$file")"
ln -s "$file" "$link"
done
zenity --info --text="Done"
fi

Rename Files by Type

2008-04-12T19:14:00.005Z

Useful when dealing with a large volume of recovered files with incorrect or non-existent filename extensions, this script renames all files in a directory and its subdirectories with an 'extension' of sorts with the description of the file type. Other search techniques and batch renamers can then look after restoring these to common three letter extensions.

ren.sh

t="/tmp/ren"
find -type f > "$t"
while read record[n]
do
mv "${record[n]}" "${record[n]}.$(file -b "${record[n]}")"
let "n+=1"
done < "$t"
rm "$t"

then, for example, to correctly name all JPEG files run (the complete list of files is too large to embed every option), where "JPEG" begins the description and "jpg" is the new extension:

find -name "*JPEG*" -print0 | xargs -0 rename 's/JPEG.*/jpg/'

Delivering Newspapers

2008-03-29T14:12:00.012Z

Small script (which installs itself as a daily cron if run with sudo) to download the latest front page of, in this example, the Evening Echo to the desktop as a nice little thumbnail pdf:

delivernewspaper

url="http://www.eveningecho.ie/pdf/front.pdf"
paper="eveningecho"

#!/bin/bash
home="/home/ian"

if [ ! -e /etc/cron.daily/delivernewspaper ]; then ln -s $(pwd)/delivernewspaper /etc/cron.daily/delivernewspaper; fi
cd $home/Desktop
wget $url -O ~/Desktop/$paper.pdf

delivernewspaper (2)
To download more than one, use a file in the home directory called ".newspapers" and this second edition of the script.

#!/bin/bash
config="/home/ian/.newspapers"
home="/home/ian/Desktop/newspapers"

if [ ! -e /etc/cron.daily/delivernewspaper ]; then ln -s $(pwd)/delivernewspaper /etc/cron.daily/delivernewspaper; fi
mkdir $home
chmod a+w $home
cd $home
wget -mnd -i $config
chmod a+w *

.newspapers (for example)

http://www.eveningecho.ie/pdf/front.pdf
http://download.guardian.co.uk/g24/Topstories.pdf
http://download.guardian.co.uk/g24/World.pdf
http://download.guardian.co.uk/g24/Media.pdf
http://download.guardian.co.uk/g24/Business.pdf

Animated GIFs

2008-03-26T18:54:00.002Z

Animated GIFs, I discovered, couldn't be easier. Open the first image in The Gimp. If it's a gif, set the image mode to RGB. Drop in the subsequent frames as new layers. Optimise for gif via the animation filter and save. Choose animation and the frame delay. Individual frame delays can easily edited later as part of the layer name in the new image file.

Montage

2008-03-26T18:42:00.006Z

I spent a long time trying to work out the best approach to recombining image tiles (such as maps, etc.). I've finally found a neat solution.

Install the ImageMagick suite of command-line graphics programs (easily done though Synaptic on Ubuntu).

The command is:

montage -geometry +0+0 -tile 10x10 tile*.gif complete.gif

Where -geometry +0+0 sets zero borders on each tile, -tile 10x10 sets a ten by ten montage layout, tile*.gif lists the collection of gif files (named in horizontal then vertical order), and complete.gif is the name of the final image.

Larger collections may have to be done in stages to stop the command from overloading the system. It can be done in a series of stripes for example.

Cal command

2008-02-29T19:38:00.003Z

Discovered the cal command today - one of those cute geeky, effective, simple commands in Linux. And even more fun, discovered the command:

cal 9 1752

to give the calendar for September 1752, the point at which the Gregorian lept 10 days over the Julian calendar.

http://news.bbc.co.uk/1/hi/magazine/7269816.stm

Linux Fonts

2008-02-11T11:34:00.000Z

Seems to be missing in Ubuntu, but a .fonts folder in the home directory allows you to copy your Windows/Wine ttf fonts over to your Linux system - and stops documents from behaving differently.

Backup Partition Table

2008-02-03T14:29:00.004Z

Where /dev/sda is the disk partition table to be backed up:

Backup

dd if=/dev/sda of=sda.mbr bs=512 count=1

To keep a human readable copy, useful when recovering from problems:

sudo fdisk -l > partitions.lst

Restore

dd if=sda.mbr of=/dev/sda bs=512 count=1

Essential before any partition editing or recovery operations which might damage the partition.

Snapshot Backup Script

2008-01-28T11:10:00.002Z

This script uses rsync to create fast and space saving dated snapshot backups. Each directory created contains a full snapshot backup but with hardlinks to the latest versions of the same files - saving time to backup and space on your backup device. It is a variation of the script described at http://www.mikerubel.org/computers/rsync_snapshots/.

today="$(date +%Y%m%d)"
source="/home/username"
backupdisk="/media/disk"
exclude="--exclude=.beagle --exclude=.Trash*"
destination="$backupdisk/backup-$today"
link="$(find $backupdisk -maxdepth 1 -type d -name "backup*" | sort | tail --lines 1)"
rsync -aHv --delete $exclude --link-dest=$link $source/ $destination/

However I experience a problem with the --link-dest option so a safer option is as follows: using the copy command to make the new directory of links.

today="$(date +%Y%m%d)"
source="/home/ian"
backupdisk="/media/disk"
exclude="--exclude=.beagle --exclude=.Trash* --exclude=*.vmdk* --exclude=*.google* --exclude=*.picasa* --exclude=*.thumbnails*"
destination="$backupdisk/backup-$today"
link="$(find $backupdisk -maxdepth 1 -type d -name "backup*" | grep -v $today | sort | tail --lines 1)"
cp -al $link/. $destination
rsync -aHv --delete $exclude $source/ $destination/

Restore All TAR Archives in a Directory

2008-01-28T11:01:00.000Z

gunzip *.tar.gz
for f in $(find -maxdepth 1 -name "*.tar" -printf "%f "); do tar -xvf $f; done

Restore Multifile TAR Archive

2008-01-28T10:36:00.000Z

gunzip *.tar.gz; tar -xvM $(echo $(find -maxdepth 1 -name "*.tar" -printf "-f %f \n" | sort))

Rsync the Wonderful

2007-10-25T18:21:00.001Z

Rsync is just a great command. Problems with a download, md5sum not correct, download interrupted, or a CD with a horrible scratch and facing into downloading the entire iso file again? Not a problem.

If repairing from a damaged CD, first copy the data to a file with dd, e.g.:

dd if=/dev/hdc of=ubuntu-7.10-desktop-i386.iso

To get a list from a download server, for example Ubuntu:

rsync rsync://ftp.esat.net/mirrors/releases.ubuntu.com/releases/

To only download the missing/replacement chunks use, for example with this Ubuntu iso file:

rsync -avLP rsync://ftp.esat.net/mirrors/releases.ubuntu.com/releases/gutsy/ubuntu-7.10-desktop-i386.iso ubuntu-7.10-desktop-i386.iso

Then burn a new CD.

Mac System 7

2007-10-15T10:45:00.000Z

Strangely enough you can run the Mac-on-a-Stick Mac OS 7 emulator on the Linux desktop via Wine: it's a Mac emulator running on Windows, running on Linux. Old Mac software is available to download at http://www.info.apple.com/support/oldersoftwarelist.html.

Running Windows

2007-08-19T11:12:00.000Z

Although so many programs are running effectively with Wine (and I can recommend Winedoors from www.wine-doors.org or Crossover Office from www.codeweavers.com) sometimes it is handy to be able to run a copy of native Windows as a virtual machine. Using VMware (www.vmware.com) is not that difficult to setup.

Make sure your kernel source and C compiler are installed.
Install VMPlayer (downloadable for free from www.vmware.com).
Run "vmware-config.pl" from the terminal as root.
Next, you need to create a virtual drive. With qemu installed, in the vmware working directory to create a drive called windowsxppro.vmdk to a limit of 2G run "qemu-img create -f vmdk windowsxppro.vmdk 2G".
Create the virtual machine as the file windowsxp.vmx as follows:

config.version = "8"
virtualHW.version = "3"
ide0:0.present = "TRUE"
ide0:0.filename = "windowsxppro.vmdk"
memsize = "64"
MemAllowAutoScaleDown = "FALSE"
ide1:0.present = "TRUE"
ide1:0.fileName = "auto detect"
ide1:0.deviceType = "cdrom-raw"
ide1:0.autodetect = "TRUE"
floppy0.present = "FALSE"
ethernet0.present = "TRUE"
usb.present = "TRUE"
sound.present = "TRUE"
sound.virtualDev = "es1371"
displayName = "Windows XP Pro"
guestOS = "winXPPro"
nvram = "WindowsXPPro.nvram"
MemTrimRate = "-1"

ide0:0.redo = ""
ethernet0.addressType = "generated"
uuid.location = "56 4d 5c cc 3d 4a 43 29-55 89 5c 28 1e 7e 06 58"
uuid.bios = "56 4d 5c cc 3d 4a 43 29-55 89 5c 28 1e 7e 06 58"
ethernet0.generatedAddress = "00:0c:29:7e:06:58"
ethernet0.generatedAddressOffset = "0"

tools.syncTime = "TRUE"
ide1:0.startConnected = "TRUE"

uuid.action = "create"

checkpoint.vmState = ""

Running vmware from the program menu you should now be able to open your virtual machine. With the operating system installation CD allow the virtual machine to boot and install Windows are normal.
Download and extract the guest operating system tools (part of the workstation tar.gz package, license number restricted but free to download the package) from www.vmware.com: in this case the file windows.iso. Extract the files to a USB drive or burn a CD and install the software in the guest operating system. This has a number of benefits as well as shared folders with the host machine, better mouse support and various controls.
Also included in the worksation tar package is the command vmware-vdiskmanager - useful for resizing your virtual drive later: e.g., "./vmware-vdiskmanager -x 20Gb ~/vmware/windowsxppro.vmdk". The Windows partition still needs to be resized on the virtual disk in the usual way - booting the virtual machine from the GParted live CD works nicely.
To create a shared folder (e.g. /home/username/shared) add the following to the windowsxp.vmx file:

sharedFolder0.enabled = "TRUE"
sharedFolder0.present = "TRUE"
sharedFolder0.writeAccess = "TRUE"
sharedFolder0.readAccess = "TRUE"
sharedFolder0.hostPath = "/home/username/shared"
sharedFolder0.guestName ="winxppro"
sharedFolder0.eXpiration = "never"
sharedFolder.maxNum = "1"

Settings: The amount of memory available for the virtual machine can be easily adjusted from the menu. I found Internet access only available using a proxy server (squid) from the host system.

Fix EXE Learning Objects

2007-08-11T18:03:00.000Z

#!/bin/bash

Version="0.80"

# Fix EXE Learning Objects (fixes problems in organisation and body tag)

# Ian Spillane, Cork, Ireland, 2007
# iantheteacher at gmail dot com
# Released under the GNU Public License Version 3.0

# See -h option for information.

InstallDirectory="$(echo ~)/bin"
HelpText="\nfixEXE (Version $Version)\nIan Spillane, Cork, Ireland, 2007\niantheteacher at gmail dot com\nFixes problems in EXE created LOs (adds beforeunload script and removes manifest error).\n\nINSTALL: ./fixEXE.sh -i\nHELP: fixEXE -h\nRUN: fixEXE.sh <zipfile>\n\t(or run as Nautilus script)\n"

case "$1" in
   '' | '-h')
       echo -e "$HelpText"
       ;;
   '-i')
       echo "Installing ..."
       cp fixEXE.sh "$InstallDirectory"
       ln -fs "$InstallDirectory/fixEXE.sh" "$(echo ~)/.gnome2/nautilus-scripts/fixEXE"
       ;;
   *)
       unzip "$1" -d "$1.files"
       sed -i 's/^<title.*$/<title>Contents<\/title>/g' "$1.files/imsmanifest.xml"
       sed -i '/^<\/item>/d' "$1.files/imsmanifest.xml"
       sed -i 's/<\/title>/<\/title><\/item>/g' "$1.files/imsmanifest.xml"
       sed -i 's/<\/title><\/item><\/item>/<\/title><\/item>/g' "$1.files/imsmanifest.xml"
       sed -i 's/<title>Contents<\/title><\/item>/<title>Contents<\/title>/g' "$1.files/imsmanifest.xml"
       cd "$1.files"
       for f in $(find -name "*.html")
       do
           echo $f
           sed -i "/beforeunload/! s/onunload=/onbeforeunload='unloadPage()' onunload=/g" "$f"
       done
       zip -r "../$1" *
       cd ..
       rm "$1.files" -r
       ;;
esac

Converting Websites to Learning Objects

2007-08-11T17:37:00.001Z

web2LO, Learning Object Packaging (Version 0.901)
http://web2lo.sourceforge.net
Developed as part of the SLOOP Project, http://www.sloopproject.eu
Released on the GNU Public License Version 3.0.

The program automatically converts a HTML website into a SCORM-compliant Learning Object: corrects DOS line breaks; corrects filename cases and spaces in URLs; adds schema files; adds javascript files; embeds scripting tags; creates files manifest and metadata tags.

web2LO runs in Linux (and other Unixes). To install, extract the zipped file web2LO.tar.gz and from the command line run "./web2LO.sh -i". The script is installed in the user's bin directory.

With simplicity in mind, the program converts a folder of website files into a packaged learning object. The folder name is used as the default title for the learning object. Chapter/SCO titles are created from existing HTML page title tags.

Organisation definition is based on filenames and is easily customised. The manifest indexes .htm/.html files in the main folder in alphabetical/numerical order (e.g. ch01.htm, ch02.htm, etc.). A single level only index is created - deeper linked files should be created in subfolders and are included in the manifest as resource files.

Run "web2LO.sh " from the command line.

Program prompts guide the user through metadata creation at package time. Metadata defaults are automatically updated in the ~/bin/web2LO/metadata.csv file (hopefully making it easier for the next package).

To customise the metadata template edit ~/bin/web2LO/metadata.csv and ~/bin/web2LO/metadata.tem (see http://www.adlnet.gov/scorm/history/Scorm12/Documents.aspx). The structure of metadata.csv is simple semi-colon (not comma) delimited text file with a single record of three fields per line: the variable name; appropriate help text; and a default value. The file metadata.tem is a template XML file. Variable names are included between hashes ("#").

To display help run "web2LO.sh -h" from the command line.

Packaged LOs have been successfully tested on Moodle (http://www.moodle.org), FreeLOms (http://www.freeloms.org), and Reload (http://www.reload.ac.uk).

The program is released on the GNU Public License Version 3.0 and listed on SourceForge at http://web2lo.sourceforge.net.

OCRPDF

2007-06-28T18:42:00.000Z

Simple script for OCR for PDF files (solves the problem of so many conversions!). Generates results from both OCRad and gOCR programs.

ocrpdf ()
{
pdftoppm $1 $1.ppm
for ppm in $(ls -1 $1.ppm*); do ocrad $ppm >> $1.ocrad.txt; gocr $ppm >> $1.gocr.txt; rm $ppm; done
}

if [ $1 = "" ]
then
for f in $(ls -1 *.pdf); do ocrpdf $f; done
else
ocrpdf $1
fi

dotTag File Auditor

2007-06-28T18:01:00.001Z

Simple extensible file auditing/security script for MD5 file signatures:

detect file integrity problems, copy/distribution flaws, file loss, security problems, and other changes;
search for duplicate files;
create a backup of MD5 signatures.

It can be run as a terminal command or as an automated cron.

The script uses a very simple but effective, reusable and extensible method to 'tag' each directory with file signatures and file change audits.

The system is capable of detecting file integrity problems (i.e. file errors), file deletions, file renames, folder renames or movements, duplicate files, and deleted folder records.

A complete backup of MD5 signatures provides a useful map to file recovery in the event of a problem. It further has the potential to provide MD5 information to other programs such as third party duplicate searches, file comparisons, and malicious file searches.

http://dottag.sourceforge.net

ECB Exchange Rates Ticker

2007-06-15T15:11:00.000Z

Small script for KNewsTicker to generate an rss feed from live ECB xml data. Edit the currency list as required.

#!/bin/bash

# ecbrss.sh, version 1, by Ian Spillane (ispillane at ccoc dot ie),
2007.
# A KNewsTicker script

# Currencies
currencies="USD|GBP"

echo "<?xml version='1.0' encoding='utf-8'?><?xml-stylesheet
title='XSL_formatting' type='text/xsl'
href='rss_convert.xsl'?><rss
version='0.91'><channel><title>ECB Reference
Rates</title><item><title>ECB
Rates</title><link>http://www.ecb.eu"

# For all lines with selected currencies change '<Cube...' to tags;
remove text; add tags to end;

wget -q -O - http://www.ecb.int/stats/eurofxref/eurofxref-daily.xml |
grep -E $currencies | perl -p -i -e "s/<Cube
currency='/<\/link><\/item><item><title>/g" |
perl -p -i -e "s/' rate='/ /g" | perl -p -i -e
"s/'\/>/<\/title><link>http:\/\/www.ecb.eu/g"

echo "</link></item></channel></rss>"

BBC News Ticker

2007-06-09T13:13:00.000Z

A script to use with KNewsTicker to supply the BBC News Ticker's headlines to the Linux desktop.

#!/bin/bash

# bbcrss.sh, version 1, by Ian Spillane (ispillane at ccoc dot ie),
2007.
# A KNewsTicker script
# Icon http://feeds.bbc.co.uk/favicon.ico

echo "<?xml version='1.0' encoding='utf-8'?><?xml-stylesheet
title='XSL_formatting' type='text/xsl'
href='rss_convert.xsl'?><rss
version='0.91'><channel><title>BBC News
Ticker</title><item><title>BBC NEWS
TICKER</title><link>http://news.bbc.co.uk"

# For all lines starting with H(EADLINE) or U(RL) change HEADLINE to
tags; change URL to tags; correct problem ampersands;
wget -q -O - http://tickers.bbc.co.uk/tickerdata/story2.dat | grep
'^[HU]' | perl -p -i -e 's/HEADLINE
/<\/link><\/item><item><title>/g' | perl -p -i
-e 's/URL /<\/title><link>/g' | perl -p -i -e
's/\&/\&amp;/g'

echo "</link></item></channel></rss>"

BBC News Screensaver

2007-06-09T13:08:00.001Z

A simple script for xscreensaver to display BBC Ticker news as a screensaver (I recommend the phosophor screensaver).

You will need to run xscreensaver from the terminal to setup the script as the text source for screensavers rather than the KDE or Gnome controls - which for some unknown reasons limit the options available. On Ubuntu both the xscreensaver application and the xscreensaver-data-extra packages (for phosphor) have to be installed.

xscreensaver

The xscreensaver package itself has some nice options, including a simple entry for a URL for a text feed in the advanced tab. For instance, http://www.rte.ie/rss/news.xml, for RTE news in Ireland works perfectly with the phosphor screensaver.

BBC

#!/bin/bash

# bbc.sh, version 1, by Ian Spillane (ispillane at ccoc dot ie), 2007.
# A script to prepare BBC News ticker for xscreensaver

# For all lines starting with H or B change HEADLINE to BBC>
wget -q -O - http://tickers.bbc.co.uk/tickerdata/story2.dat | grep '^[HB]' | perl -p -i -e 's/HEADLINE/BBC>/g'

System Ticker

2007-06-09T12:33:00.000Z

A simple script to be used by KNewsTicker to provide system uptime information. Can be used alongside online news or as an entirely offline ticker. Has the potential to be easily adapted to provide more information.

#!/bin/bash
# systemrss.sh, version 1, by Ian Spillane (ispillane at ccoc dot ie),
2007.
# A KNewsTicker script

echo "<?xml version='1.0' encoding='utf-8'?><?xml-stylesheet
title='XSL_formatting' type='text/xsl'
href='rss_convert.xsl'?><rss
version='0.91'><channel><title>System
Information</title><item><title>"

uptime

echo "</title><link>sysinfo:/</link></item></channel></rss>"

X Configuration Problems

2007-05-31T10:29:00.000Z

I had a problem recently on OpenSuse 10.2 when changing to a new LCD widescreen monitor. Of course I screwed up the xorg.conf settings and X would no longer start. There is I'm sure a straightforward way to fix this but I was saved by the Ubuntu CD.

Boot with Ubuntu, mount the harddisk system partition, and copy the xorg.conf file from Ubuntu to my OpenSuse 10.2 installation.

Check for Website Changes

2007-05-24T13:13:00.001Z

I had a problem with hacking on a commercial website recently and instead of constantly checking I wrote a little script (for which I put a symbolic link in /etc/cron.hourly) to automate the job and send me an email to my phone if there was any change to the published site.

site=sitename.com
email=recipient@email.com

checksum=$(cat /tmp/"$site.chk")
wget -q http://$site -O /tmp/"$site.tmp"
echo $(cat /tmp/"$site.tmp" | md5sum) > /tmp/"$site.chk"
rm /tmp/"$site.tmp"
latest=$(cat /tmp/"$site.chk")
if [ "$checksum" != "$latest" ]
then
echo "$site $latest" | /usr/lib/sendmail $email
fi

Dark Monitor

2007-05-09T01:06:00.000Z

To fix a problem with an old dark monitor I've added the simple command:

xgamma -gamma 1.3

to Sessions > Startup Programs in the Control Center. 1.3 is a value brighter than 1.0. Different values can be easily tested at the command prompt.

Or at the command line one can add the autostart file directly via:

echo "[Desktop Entry]
Name=xgamma
Encoding=UTF-8
Version=1.0
Exec=xgamma -gamma 1.3
X-GNOME-Autostart-enabled=true" > ~/.config/autostart/xgamma.desktop

Could we not have this feature easily added to the desktop's monitor configuration menu?

Dell Inspiron Wireless Card on Linux

2007-04-29T11:35:00.000Z

How to configure a Dell Inspiron (6400) laptop's wireless card (Broadcom 1390) under Linux (OpenSuse 10.2):

Install ndiswrapper (included in the OpenSuse 10.2 distro and others, but more than likely not installed by default).
Prevent the system from attempting to use the bcm43 driver. As SU at the terminal, enter:

echo "blacklist bcm43xx" >> /modprobe.d/blacklist
With access to the original Windows drivers for your system.

ndiswrapper -i bcmwl5.inf
To check what drivers are installed with ndiswrapper then:

ndiswrapper -l

expecting:

bcmwl5 driver installed, hardware (14E4:4328) present
To load (first unloading) the ndiswrapper driver into Linux:

rmmod ndiswrapper
modprobe ndiswrapper
To check that everything has been recognised OK and read system messages:

dmesg | grep ndiswrapper
dmesg | grep wlan
To check that your wireless card is working:

iwlist scanning
To make sure that ndiswrapper is loaded permanently on boot go into YaST > /etc/sysconfig editor. Go into the tree of settings and browse to System > Kernel > MODULES_LOADED_ON_BOOT and add 'ndiswrapper'.
To setup your network card in YaST for use on the desktop with Network Manager, etc., go to Networking Devices > Network Card and add your new card with the hardware set as "wlan0" and the driver as "ndiswrapper".

Internet Connection Sharing

2007-04-11T18:59:00.001Z

You would think it would be obvious - sharing an Internet connection between two or more networked computers on Linux. It wasn't to me. But annoyingly, it then had such a simple solution that one would think Suse's Yast could have enough manners to make it all happen with a simple button.

Make sure Squid (the proxy server) is installed. It is a part of the distro in OpenSuse 10.2 and I assume any distro.

Edit the /etc/squid/squid.conf file (as superuser) to uncomment, and edit if necessary, down in the custom settings section the lines which read something like:

acl our_networks src 192.168.1.0/24
http_access allow our_networks

Change the "12.168.1.0/24" to match your network requirements.

Webcam Blogging Script

2007-03-06T01:11:00.001Z

This script, installed as an hourly cron job, simply takes a webcam shot and emails it to a blogging photo album - in this case flickr.com which provides a unique email address to email images directly to the photo album and automates the process of posting new photos directly to a blog such as blogger.com.

The script uses the mailer script mail_files.sh (available at http://jazstudios.homelinux.net/main/trans/mail_script/) to send the file to the required email address.

The command streamer is used to capture the image (available as part of the xawtv package). The -w option forces it to wait long enough for the auto exposure to adjust on my camera, the -q sets it to run in quiet mode.

This script is currently in action posting hourly webcam shots of my front yard to webcamshots.blogspot.com.

Installation steps:

Change the path in the script to the script file mail_files.sh (available to download from
Change the paths in the script for the location of the local photo album to keep in the images in.
Replace the [uniquecode] in script email address [uniquecode]talk2blog@photos.flickr.com with the one assigned to you by flickr.com.
Install the script as a cron job - one way is as su to create a symbolic link from the script in the folder /etc/cron.hourly.

#!/bin/bash

# snap.sh, version 1, by Ian Spillane (ispillane at ccoc dot ie), 2007.
# A script to email an automated webcam shot.
# Released on the GPL (http://www.gnu.org/copyleft/gpl.html).

# Uses the script mail_files.sh, available at http://jazstudios.homelinux.net/main/trans/mail_script/

d=$(date +%Y-%B-%d)
t=$(date +%H:%M)

mailer=/home/ian/admin/scripts/webcam/mail_files.sh
album=/home/ian/album/webcam/

streamer -w 30 -q -o $album$d,$t.jpeg
$mailer -t [uniquecode]talk2blog@photos.flickr.com -s "Shot at $t" -m "Webcam shot at the window for $d, $t." -b "$album$d,$t.jpeg"

Vodafone 3G Modem on OpenSuse 10.2, 10.3

2007-02-28T12:29:00.001Z

The Vodafone 3G Modem (Huawei E220) is installed on a USB connection. It provides an always-on broadband connection at up to 1.4Mbps on the 3G network or falls back to dialup speeds on the lesser GPRS (2.5G) network.

No support is offered for Linux, but it does work on Linux.

OpenSuse 10.3

Modem is detected automatically.

OpenSuse 10.2

The only problem is that as a multi-function device its USB storage capability is recognised by Linux's HAL and it doesn't go any further to set up the device as a USB serial port.

A few simple fixes exist: As SU add the following lines to the configuration files.

To /etc/modprobe.conf.local add a line "options usbserial vendor=0x12d1 product=0x1003".
To /etc/rc.d/after.local add the line, or create the file if necessary, "modprobe usbserial".

Dialer

The modem is available as device /dev/ttyUSB0 and can be easily configured using wvdial or kppp.

Kppp does need the suid bit set with the command from the root terminal:

chmod u+s /opt/kde3/bin/kppp

Modem settings are:

Tone dialing
921600 baud
Init1 = ATZ
Init2 = AT +CGDCONT=1,"IP","isp.vodafone.ie" Q0 V1 E1 S0=0 &C1 &D2 +FCLASS=0 (or hs.vodadone.ie, or whatever is used)
Phone number = *99#
Username = vodafone
Password = vodafone
Modify DNS when connected
Automatically retrieve dns
Dynamic IP address

Firmware Upgrade

To fix the firmware for Windows Vista see http://www.mobileconnected.co.uk/2007/05/25/usb-modem-huawei-e220-on-vista-take-2/ however note that the dialup number is now "*99***16#".

Sync Folder (for USB drives, etc.)

2007-01-16T15:39:00.007Z

A Nautilus script to automatically sync a folder on a USB drive or elsewhere with a folder of the same name in the home folder. The script will not upload files with "archive", "resources" or "backup" to the remote disk. Add the extra extension .delete to any remote files you want to delete and also want deleted back in the local home folder (otherwise deleted files will keep coming back in future syncs). (for snapshot backup to USB or other drive, see this post)

syncfolder

#!/bin/bash

# SyncFolder, version 1.4, by Ian Spillane (ispillane at ccoc dot ie), 2007.
# A Nautilus script to automatically sync a folder on a USB drive or elsewhere with a folder of the same name in the home folder.
# Released on the GPL (http://www.gnu.org/copyleft/gpl.html).

# To install, run the script.
# To sync, right-click on the remote folder in nautilus and select syncfolder.
# To delete a file remotely and have the deletion repeated, rename the file to whatever.delete.

remote="$*/"
local="$HOME/$*/"
date=$(date +%y%m%d)
exclude="--exclude=*archive/* --exclude=*resources/* --exclude=*backup/* --exclude=*Thumbs.db*"

if test -z "$*"
then
# If no arguments: install as Nautilus script
cp syncfolder ~/.gnome2/nautilus-scripts/syncfolder
echo "Installed as Nautilus script. To use, right-click on a folder in Nautilus and select scripts."
else
echo ------------------------------ >> sync.log
echo >> sync.log
date >> sync.log
rsync -vazrut --modify-window=1 -b --backup-dir="$local/backup/$date" --exclude=*.delete "$remote" "$local" >> sync.log
cd $remote
find -name *.delete -printf "%P\n" | while read record[f]
do
rm -v "${record[f]}"  >> ../sync.log
rm -v "$local/$(echo ${record[f]} | sed s/.delete//g )"  >> ../sync.log
rm -v "$local/$(echo ${record[f]})"  >> ../sync.log
done
cd ..
rsync -vazrut --modify-window=1 --delete $exclude "$local" "$remote"  >> sync.log
zenity --text-info --filename="sync.log" &
fi

I use it all the time so that I can keep my school work folder on a USB and not worry about working at home and at work and where the latest file is. Some errors have occurred but only caused by a full drive or me forgetting to sync before re-editing the wrong edition of the file. But mostly it has been a great solution for me. The built-in backup facility saves any disaster copies from the remote folder to the local. I even worked out how to delete a file on the USB and have it deleted when I go home. Renaming files doesn't really work, but you can't have everything.

I would enjoy feedback on the script and help with it. Wouldn't it be wonderful to have the function really embedded in Nautilus and automatically activated on mounting of the USB.

(For a great tool to keep machines synchronised see Dropbox.)

Fix Website URLs

2007-01-13T19:54:00.000Z

While a lot of materials will have been created in Windows, which is case-insensitive, Linux and Linux powered webservers are case sensitive (distinguishing between uppercase and lowercase letters in filenames) and have problems with spaces in filenames.

This script fixes website URLs for Linux, fixing DOS line breaks, renaming all files to lowercase and converting all a, img, frame, and script tags in HTML and resource, file, and item tags in XML files (SCORM LOs) to lowercase.

Use it as a regular script or install it as a Nautilus script to automatically add a right-click script option to fix the URLs in a complete folder. Installs as a Nautilus script if run without argument.

fixURLs.sh

>#!/bin/bash

# fixURLs, version 1.1, by Ian Spillane, 2007.
# http://iantheteacher.blogspot.com (ispillane at ccoc dot ie).

# A bash script to automatically change all site URLs to lowercase and remove spaces from file names.
# Installs as a Nautilus script if run without argument.
# Released on the GPL (http://www.gnu.org/copyleft/gpl.html).

if test -z "$*"
then
# If no arguments: install as Nautilus script
cp fixURLs.sh ~/.gnome2/nautilus-scripts/fixURLs
echo "Installed as Nautilus script. To use, right-click on a folder in Nautilus and select scripts."
else
original=$(echo "$*" | sed s/\\///)
fixed=".$original"

# create new directories
mkdir "$fixed"
for file in $(find "$original" -type d | sort | sed s/[\ ]/{}/g)
do
f=$(echo $file | sed s/{}/\ /g ) # return spaces
lf=`echo $f | tr '[A-Z]' '[a-z]' | tr ' ' '_'`
mkdir "$fixed/$lf"
done

# rename all files to lowercase and remove spaces
for file in $(find "$original" -type f | sort -r | sed s/[\ ]/{}/g)
do
f=$(echo $file | sed s/{}/\ /g ) # return spaces
lf=`echo $f | tr '[A-Z]' '[a-z]' | tr ' ' '_'`
cp "$f" "$fixed/$lf"
done

for file in $(find "$fixed" | grep -e .htm -e .xml | sed s/[\ ]/{}/g)
do
f=$(echo $file | sed s/{}/\ /g ) # return spaces

# correct DOS linebreaks
dos2unix -q "$f"

# convert HTML tags (a, img, frame, and script) to lowercase
sed -i 's/<A /<a /g' "$f"
sed -i 's/<FRAME /<frame /g' "$f"
sed -i 's/<SCRIPT /<script /g' "$f"
sed -i 's/<IMG /<img /g' "$f"

# convert XML (SCORM LOs) tags (item, resource, file) to lowercase
sed -i 's/<RESOURCE /<resource /g' "$f"
sed -i 's/<FILE /<file /g' "$f"
sed -i 's/<ITEM /<item /g' "$f"

# convert all URLs to lowercase
sed -i '/<a/ s/<a /\n<lowercase<a /g' "$f"
sed -i '/<frame/ s/<frame/\n<lowercase<frame/g' "$f"
sed -i '/<script/ s/<script/\n<lowercase<script/g' "$f"
sed -i '/<img/ s/<img/\n<lowercase<img/g' "$f"
sed -i '/<resource/ s/<resource/\n<lowercase<resource/g' "$f"
sed -i '/<file/ s/<file/\n<lowercase<file/g' "$f"
sed -i '/<item/ s/<item/\n<lowercase<item/g' "$f"
sed -i '/<lowercase/ s/>/><lowercaseend\n/' "$f"
sed -i '/<lowercase/ y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' "$f"

# remove spaces from URLs
for sfile in $(find "$fixed" | grep " " | sed s/[\ ]/{}/g)
do
 s=$(echo $sfile | sed s/{}/\ /g ) # return spaces
 sed -i '/^<lowercase.*$/ s/ /_/g' "$f"
done
sed -i 's/<lowercase//' "$f"
sed -i 's/<lowercaseend//' "$f"

# remove blank lines
cat "$f" | grep -v "^[ \t]*$" > "$f.tmp"
mv "$f.tmp" "$f"
done

folder=$(find "$fixed" -maxdepth 1 -mindepth 1 | sed s/"$fixed"\\///)
mv "$fixed/$folder" "$folder.fixed"
rm -r "$fixed"
fi

Autofill-Folder Nautilus Script

2007-01-10T20:04:00.001Z

A small script to make simple search folders in Nautilus - an implementation of virtual folders - ideal for creating single folders of links for documents, images or music files based on keywords.

Autofill-Folder.sh


#!/bin/bash

# Autofill-Folder, version 1, by Ian Spillane (ispillane at ccoc dot ie), 2007.
# A Nautilus script to automatically populate a folder with links to files matching the folder name. A simple but effective implementation for virutal folders for music, images, or any file type.
# Released on the GPL (http://www.gnu.org/copyleft/gpl.html).

# To do: Implementation of further search options, multiple key words, file types, etc., still based on a simple syntax in the folder name, e.g. black|white[image].

search=$*
count=0

if test -z "$*"
  then
    # If no arguments: install as Nautilus script
    cp Autofill-Folder.sh ~/.gnome2/nautilus-scripts/Autofill-Folder
    echo "Installed as Nautilus script. To use, right-click on a folder in Nautilus and select scripts."
  else
       cd $search
       # Make links in folder for every file matching name, except for .Trash
       for f in $(find ~ | grep -i "$search" | grep -v "$(pwd)" | grep -v ".Trash" | sed s/[\ ]/{}/g) 
       do
      file=$(echo $f | sed s/{}/\ /g ) # return spaces
             link=$(basename "$file")
             ln -s  "$file" "$count-$link"
             count=$[$count+1]
       done
fi