Monday 8 June 2009

Quick tips for Ubuntu 9.04

One of the beauties of starting your OS afresh is, well, you get a new and better OS. Ubuntu 9.4 (or 'jaunty jackalope') is a slick version of my previous Ubuntu 8.04 ('hardy heron'). So far, I've ascertained that it looks better, it has better support for hardware, and it boots really fast.

One of the pains of starting your OS afresh is having to reinstall all your software and set your preferences. I thought I'd put a little 'as I do it' guide which may take some of the pain away. This list has a few of the 'biggies' and maybe a couple of items more suited to my personal taste, but what the hey!

1. configure the firewall
sudo apt-get install gufw

2. install wine (windows emulator, for those windows programs you cant live without - an ever decreasing number horray!)
sudo apt-get install wine

3. This one is crucial if you want to run java - it takes ages, and at the end when it installs java, you need to be able to use the tab and 'pg dwn' keys to get to the end of the license agreement
sudo apt-get install ubuntu-restricted-extras

4. VLC is still the best media player around
sudo apt-get install vlc

5. skype -
sudo wget http://www.medibuntu.org/sources.list.d/jaunty.list -O /etc/apt/sources.list.d/medibuntu.list

wget -q http://packages.medibuntu.org/medibuntu-key.gpg -O- | sudo apt-key add - && sudo apt-get update

sudo aptitude install skype

6. Google Earth (version 5) - install the binary from here. Right click - Properties - Permissions - make executable. Then double click, and it will install - easy! It will create a desktop icon which, by default, you cannot see because you do not have permission. In the terminal type (here, replace 'daniel' with your username):

cd ./Desktop
sudo chown daniel:daniel Google-googlearth.desktop

7. Expand the capabilities of nautilus file browser

sudo apt-get install nautilus-actions nautilus-gksu nautilus-image-converter nautilus-open-terminal nautilus-script-audio-convert nautilus-script-collection-svn nautilus-script-manager nautilus-sendto nautilus-share nautilus-wallpaper

8. Adobe AIR

wget http://airdownload.adobe.com/air/lin/download/latest/AdobeAIRInstaller.bin
chmod +x ./AdobeAIRInstaller.bin
./AdobeAIRInstaller.bin

9. Adobe acrobat reader

sudo gedit /etc/apt/sources.list

Add the below line in sources.list

deb http://archive.canonical.com/ubuntu jaunty partner

sudo apt-get update
sudo apt-get install acroread

G'night!

Sunday 7 June 2009

Never do this

 sudo mv filename.file /bin/sh  


Just don't mess with your bin/sh, and secondly don't attempt to restart your computer after you mess with your /bin/sh (which is the symbolic link to /bin/dash - low level crazy stuff which ordinarily you shouldnt ever know or care about, but which ensures your computer can do some vital things, like initialise).

This I discovered to my cost, and now I'm having to spend my weekend recovering all my files

This is a worrying and heartbreaking exercise, as I'm sure you'll understand - my computer is my livelihood and most of my hobbies, and I havent backed stuff up for ages, and recently I've been doing some cool stuff which exists only on the computer which now needs a complete OS overhaul!!

I'm writing this for instruction and warning rather than pity, you understand. You can't reboot, not even in any safe mode. I did a bit of internet forum research, and there was little guidance (mostly, instructions for windows users to use Ubuntu to recover their files, but not Ubuntu users to recover their files)

It turns out it's thankfully simple.

1. download an Ubuntu distro iso, and burn it to CD (instructions here and here)
2. place in optical drive of bummed out computer, and use BIOS to instruct it to boot from disk
3. select option to 'try Ubuntu without installing it on your computer' (or whatever the exact wording is - should be the first option in most versions)
4. then wait for it to boot - the OS running from the CD can recognise your hard-drive, but won't let you automatically access it. Open a terminal and type the following (assuming you want to access drive sda1 - you may want sdb1 or something):

 sudo mkdir /media/ubuntu  
sudo mount -t ext3 /dev/sda1 /media/ubuntu


for ease of copying and later access, change the permissions to all directories so anyone can have read/write privileges:

 sudo su  
chmod 777 -R /media/ubuntu


this will take a while if you have a lot of files and directories, but when it's done you're ready to plug in a large external hard drive, and save your precious files away - exactly what I'm doing now! The next step will be to reinstall ubuntu (I'm going to go for the latest - 9.04) and spend weeks reinstalling all my software and getting it to my specification - weeks I really don't have!

Friday 5 June 2009

Flower Bed

Recently I investigated Apollonian gaskets (a type of space-filling fractal composed of ever-repeating circles) for use as 'pore-space fillers' in a model I am creating for synthetic sediment beds. I came across a most excellent script here:

http://www.mathworks.com/matlabcentral/fileexchange/15987

In my Friday afternoon boredom, I have just modified some aspects of the program to create a function which simulates a pretty flower bed Some example outputs below - enjoy! Right, now back to work ...






Thursday 16 April 2009

Frequency Transforms on Images

As part of my work, I have to do a lot of spectral analysis with images. Traditionally I have used MATLAB to do this - it is exceptionally easy and powerful, and works just fine. As a simple example, if I want a power-spectrum of an image, treated as a 2D matrix of 8-bit numbers, the sequence of commands is this:

 %------------------- MATLAB  

% read image in, convert to greyscale and double-precision
image=double(rgb2gray(imread('myimage.jpg')));
% crop to smallest dimension
[m,n]=size(im);
im=im(1:min(m,n),1:min(m,n));
% find image mean
imMean=mean(im(:));
% and subtract mean from the image
demean=im-imMean;
% find the 2D fourier transform
ft=fft2(demean);
% power spectrum is the element-by-element square of the fourier transform
spectrum=abs(ft).^2;


Seven lines of commands. Couldn't be simpler. That's the beauty of MATLAB. The snag is that it costs several hundreds to thousands of dollars for the program, which makes it difficult to share your code with other people. So recently I've thought about coding up some things I want to share with other people in Python.

Python is a cross-platform, open-source programming language and environment. It is not too dissimilar to MATLAB in many respects, especially when you install the right libraries. In order to do the same in Python as above, as easy as possible, I needed to install Python Imaging Library (PIL),  NumPy and SciPy. Then the sequence of operations is as follows:


 # ----------------PYTHON  

#-----------------------
# declare libraries to use
import Image
import scipy.fftpack
from pylab import*
import cmath
from numpy import*
# load image, convert to greyscale
im=Image.open('myimage.jpg').convert("L")
# crop to smallest dimension
box=(0,0,im.ny,im.nx)
region=im.crop(box)
# find mean
imMean=mean(asarray(region))
# 2D fourier transform of demeaned image
ft=fft2(region-imMean)
# power spectrum
spectrum=pow(abs(ft),2)


Same answer. Not bad - not harder, just different. And completely free. Oh, and this'll work on UNIX, Linux, Mac, and PC when Python is installed (with a couple of extra libraries). Great! 

Hmm, I thought, I wonder if it is possible to do the same in a piece of non-numerical software. I tried ImageMagick because its free and ubiquitous on Linux and Mac, and also available for PC, and it has a massive user-community. It is incredibly powerful as a command-line graphics package - even more powerful than PhotoShop and Illustrator, and like all UNIX-based applications, infinitely flexible. It turns out it is a little more tricky to calculate the power spectrum in ImageMagick, but here's how.

First, you need the right tools - ImageMagick is usually already present on most Linux distributions - if not, if a debian/Ubuntu user use sudo apt-get install imagemagick, or download from the webiste hyperlinked above. Then you'll need an open-source C-program for calculation of FFTs: this is provided by FTW - download fftw-3.2.1.tar.gz, untar, cd to that directory, then install the binary by issuing the usual commands:

 ./configure  

sudo make && make install


This C-program is no good on its own, so the last ingredient is a wrapper for ImageMagick. Fortunately - nay, crucially, this has been provided by Sean Burke and Fred Weinhaus - introducing IMFFT! The program can be downloaded and installed from here. And here is an in-depth tutorial of how to use it (I'm only just getting to grips with it myself). For some reason the compiled program is called 'demo'. I move it to my /bin directory so my computer can see it wherever I am working, and in the process change the name to something more suitable - fft:

 cp demo ./fft && sudo mv ./fft /bin  



Right, it's a little bit more involved, but here is the bash shell script for the power spectrum in ImageMagick:

 #----------------------------- IMAGEMAGICK  

#-------------------------------------------------
#!/bin/bash
# read image and convert to greyscale
convert myimage.tif -colorspace gray input_grey.tiff
# find min dimension for cropping
min=`convert input_grey.tiff -format "%[fx: min(w,h)]" info:`
# crop image
convert input_grey.tiff -background white -gravity center -extent ${min}x${min} input_grey_crop.tiff
# de-mean image
# get data
data=`convert input_grey_crop.tiff -verbose info:`
# find mean as proportion
mean=`echo "$data" | sed -n '/^.*[Mm]ean:.*[(]\([0-9.]*\).*$/{ s//\1/; p; q; }'`
# convert mean to percent
m=`echo "scale=1; $mean *100 / 1" | bc`
# de-mean image
convert input_grey_crop.tiff -evaluate subtract ${m}% ggc_dm.tiff
# pad to power of 2
p2=`convert ggc_dm.tiff -format "%[fx:2^(ceil(log(max(w,h))/log(2)))]" info:`
convert ggc_dm.tiff -background white -gravity center -extent ${p2}x${p2} ggc_dm_pad.tiff
# forward fast fourier transform
fft f ggc_dm_pad.tiff
# creates ggc_dm_F_Q16.tiff, on which get spectrum
convert ggc_dm_F_Q16.tiff -contrast-stretch 0% -gamma .5 -fill "rgb(1,1,1)" -opaque black ggc_dm_F_Q16_spec.tiff


Voila! The spectrum is the image 'ggc_dm_F_Q16_spec.tiff'. I haven't yet worked out how to keep all the data inside the program, to eliminate the need (and cumulative errors associated with) writing the outputs at every stage to file, and eading them back in again. This means I can't use the spectrum for any scientific application. But in theory it should be possible. Any suggestions welcome!

Saturday 28 February 2009

Shannon's Box

Claude Shannon, father of information theory, apparently once invented a box whose sole purpose was to, when switched on, turn itself off again. Recently I've got into scripting with CHDK (Canon Hackers Development Kit) - a firmware enhancement for Canon cameras. It allows you to turn your cheap point-and-shoot camera into something better, for free. Long exposures, very rapid shots, motion detection - you name it. I can tell I'm at the start of a beautiful friendship ... look at this!

You can write and upload scripts to automate processes. The language is uBASIC - really intuitive and easy. My first script is pretty basic - it has been to tell the camera to, when switched on, automatically take 3 photos and then turn itself off! I therefore called it Shannon's Box:

 rem Author Daniel Buscombe  
@title Shannon's Box
@param a Shoot Count
@default a 3
shoot full
for n=2 to a
shoot full
if n=a then goto "shutdown"
next n
end
:shutdown
shut_down


More, I'm sure, on CHDK in the following months.

Wednesday 25 February 2009

Ubuntu Beautification

Tip #1:

It's amazing really - most fonts contain within them “hints” laid down by their designer about how they should look on-screen. Artistic meta-data! However, Ubuntu ignores them and uses a system called 'autohinting'.

It works well, but you might also want to try bytecode hinting. This uses the hinting built into the fonts. To activate bytecode hinting, open a terminal window and type:

$ sudo dpkg-reconfigure fontconfig-config

Using the cursor keys, select Native from the menu and then hit Enter.

On the next screen you’ll be asked if you want to activate subpixel rendering. This is good for TFT screens, so make the choice (or just select Automatic). Next you’ll be asked if you want to activate bitmap fonts, which are non-true type fonts good for use at low point levels. There’s no harm in using them, so select yes. The program will quit when it’s finished. Once that’s happened, type the following to write the changes to the system and update files:

$ sudo dpkg-reconfigure fontconfig
$ sudo dpkg-reconfigure defoma

System - Quit - Logout, and then log back in again. Letters will appear more rounded and the antialiasing will appear better.

Tip #2:

465 fonts of Brian Kent (http://www.aenigmafonts.com) are available to Ubuntu users. To install the fonts, you’ll need to add a new software repository:

System - Administration - Software Sources
- Third-Party Software tab - Add

type the following into the dialog box that appears:

deb http://ppa.launchpad.net/corenominal/ubuntu hardy main

Click 'Add Sources', then Close and, when prompted, agree to reload the package lists. Then use Synaptic to search for and install the 'ttf-aenigma' package. Once installed the fonts will be available for use straight away in all applications.

Enjoy your beautified Ubuntu!

How to get on youtube, twitter and myspace quicker

Well, the easiest way is save time when you boot your computer!

Whenever Ubuntu boots it runs several scripts that start necessary background services. By default these are set to run one-by-one but if you have a processor with more than one core, such as Intel’s CoreDuo series or AMD’s Athlon X2, you can configure Ubuntu to run the scripts
in parallel. This way all the cores are utilized and quite a bit of time can be saved at each boot. To make the change, type the following to open the necessary configuration file in Gedit:

$ gksu gedit /etc/init.d/rc

Look for the line that reads CONCURRENCY=none and change it so it reads CONCURRENCY=shell.

Then save the file and reboot your computer. This should save you several seconds of time every time you boot up, and you can go on being a handsome speedy devil

Every pixel tells a story

Recently the Daily Daniel ran a story (hark at me! I'm not The Times!) with a single photograph - a mosaic - each 'pixel' containing a separate photo. I've finally got round to explaining how it is done, if only to show that I didn't arrange the pictures manually - she is beautiful, admittedly, but I gotta work to eat!

Right, the Linux program is called 'metapixel' and is downloaded from here. If you've enough photographs, you can produce some incredible results

You need to issue 2 commands - one prepares the photos, then the other makes a mosaic out of them. Create a folder somewhere to store the prepared photos called, say, 'recast'

$ metapixel-preparee -r /directory/where/photos/are /directory/recast --width=40 --height=40

The larger the width and height, the longer it takes. Be sure to get the aspect ratio of the photos right - it doesnt check or automatically correct. Choose a single photo that you want to make a mosaic out of (e.g. /directory/in.png), then (be warned, this may/will take several hours for a big photo!):

$ metapixel /directory/in.png --library /directory/recast --scale=5 --distance=5

The scale tells it how big you want the output (i.e. here 5 x size of input image), and the distance tells it how far to look for matches - it matches by colour so its a really good idea to scale by the range of inputs. It will give you an error if the distance parameter is too large. The results can be mixed, but once you hone the inputs and parameters down, and you have a LOT of patience, you can get some really great mosaics!

Friday 13 February 2009

Tweetdeck on ubuntu 64-bit

Right, the whole world's into twitter. The website is a little difficult to manage when you have a lot of people to follow. The software solution (isn't there always one!) is TweetDeck - unfortunately it runs under Adobe Air, which (surprise, surprise) is available for Linux but 64-bit architectures. So your computer requires a little bit of persuading ...

... you need a few libraries - issue the following commands at the terminal:
$ sudo apt-get lib32asound2 lib32gcc1 lib32ncurses5 lib32stdc++6 lib32z1 libc6 libc6-i386
$ sudo getlibs -l libnss3.so.1d
$ sudo getlibs -l libnssutil3.so.1d
$ sudo getlibs -l libsmime3.so.1d
$ sudo getlibs -l libssl3.so.1d
$ sudo getlibs -l libnspr4.so.0d
$ sudo getlibs -l libplc4.so.0d
$ sudo getlibs -l libplds4.so.0d
$ sudo getlibs -l libgnome-keyring.so
$ sudo getlibs -l libgnome-keyring.so.0
$ sudo getlibs -l libgnome-keyring.so.0.1.1
$ sudo ldconfig

Then install Adobe Air from here

download, then move to your home directory,

right click - properties - make executable

and type

$ chmod +x AdobeAIRInstaller.bin
$ sudo ./AdobeAIRInstaller.bin

Then grab the 'air' file from here, and install using Adobe Air

Happy tweeting!

Monday 9 February 2009

Miscellaneous stuff of use

The documentation on the use of 'wget' is a little sketchy on one of its most powerful uses - automatically downloading loads of files from internet servers. Let's say you need the 1000 tiff files sitting at http://www.server.com/hypothetical_directory

Apparently wildcards are out, such as:

$ wget http://www.server.com/hypothetical_directory/*.tiff

because html does not support 'globbing'. So here's the solution:

$ wget -r -1l --no-parent -A.tiff http://www.server.com/hypothetical_directory

%----------------------------------
Another thing I've needed to do recently is get ALL the metadata from an image file - you wouldn't believe how much information is recorded - channel statistics, histogram of shades, date/time, camera make, resolution - you name it, its recorded. It's a wonderful thing about using images for your research as well - you can keep track of all your metadata in the data files! Right, well I discovered that no amount of 'right-clicking' on the image - in any operating system - will tell you everything you need to know. The suitable command is:

$ identify -verbose myimage.jpg

%------------------------------------
Finally, for now, a couple of sites that will help you if you need to make a purchase of hardware for your linux machine:

This one keeps a database of usb-devices which are known to be linux-friendly, and the following two help you out when choosing a compatible printer and scanner: here and here.

Keep on truckin'

Friday 16 January 2009

How to read and search your blog page with the imaginative use of the 'regexp' in matlab

Just a quickie - there may be some application for this (?)

You start with your blog url:

>> url='http://thezestyblogfarmer.blogspot.com/';

read it in, then you can start searching through its contents:

>> text=urlread(url);

for example, I can list all the unix commands I have mentioned - its easy because they all start with a dollar sign

>> text_linux=regexp(text,'>\$[^<\r\n]*<','match')' which gives me the output:

'>$ sudo apt-get update<' '>$ sudo apt-get upgrade<' '>$ sudo apt-get clean<' '>$ sudo apt-get autoclean<' '>$ sudo apt-get -f install<' '>$ sudo badblocks -v /dev/sda1<' '>$ sudo aptitude install debsums<' '>$ debsums <' '>$ sudo apt-get install firestarter<' '>$ sudo apt-get install youtube-dl<' '>$ youtube-dl http://uk.youtube.com/watch?v=r9OjoPskf_c<' '>$ ffmpeg -i r9OjoPskf_c.flv people_everyday.avi<' [1x97 char] '>$ sudo su | sudo apt-get install skype<' '>$ sudo jhas -jh 837afm$^&qeiuhn>>KOUUIG4n we8f-&hcjku8hujbn ok?<' '>$ sudo su | python setup.py install<' [1x75 char] '>$ sudo dpkg -i skysentials_1.0.1-1_all.deb<' '>$ konsole<' or maybe the matlab commands which all start with 2 > signs:

>> text_matlab=regexp(text,'>\>>[^<\r\n]*<','match')'

'>>> cf=0; %current frame<' '>> for k=10:50 % identity matrix from [10,10] to [50,50]<' '>> clf;<' '>> plot(fft(eye(k))) %plot<' '>> axis equal; axis off; axis([-1 1 -1 1]); % sort out axes<' '>> pause(0.01) %take a break<' '>> cf=cf+1; %update current frame<' [1x87 char] '>> end<' [1x94 char] '>>KOUUIG4n we8f-&hcjku8hujbn ok?<' '>>> convert my_image.jpg -resize 200% my_new_image.jpg<' '>>> direc=dir([pwd,filesep,'*.','jpeg']);<' '>>> filenames={};<' '>>> [filenames{1:length(direc),1}] = deal(direc.name);<' '>>> filenames=sortrows(char(filenames{:}));<' '>>> mkdir([pwd,filesep,'james_and_the_giant_peach'])<' '>>> mkdir([pwd,filesep,'thumblina'])<' '>>> for i=1:size(filenames,1)<' '>>> system(['convert ',deblank(filenames(i,:))...<' '>>> system(['convert ',deblank(filenames(i,:))...<' '>>> end<' '>>> url='http://www.mathworks.com/moler/ncm.tar.gz';<' '>>> gunzip(url,'ncm')<' '>>> untar('ncm/ncm.tar','ncm')<' '>>> cd([pwd,filesep,'ncm'])<' '>>> [U,G]=surfer('http://www.thedailydanielblog.blogspot.com',200);<' '>>> fid=fopen('dansweb.txt','wt');<' '>>> for i=1:size(char(U),1), fprintf(fid,'%s\n',char(U(i,:))); end<' '>>> fclose(fid)<' '>>> pagerank(U,G)<' and finally how to list the websites you refer to - I can't post the code into my blog because it involves html code which blogger doesn't like - replace '>\>>' with '< [the first letter of the alphabet] href'


enjoy!

The big clean

One of the beauties of a Linux system is that it is so easy to keep current. The software's free, and it tells you automatically if there are upgrades available. I probably upgrade my ubuntu hardy heron (8.04) OS every couple of days!

It's just as easy to clean, so there isn't loads of junk lying about everywhere - a problem which just kills windows machines after a while. I've had this OS since it came out (on the 4th month of 2008, hence 8.04) so I thought it was high time to have a bit of, well considering the decent run of weather we've been having here, a spring clean

First, how to upgrade and clean up the software packages. This is a crucial step for someone like me who spends a lot of time downloading and checking out new software

$ sudo apt-get update

$ sudo apt-get upgrade (the only thing I installed was 'replacement guidance-backends' - whatever they are!)

$ sudo apt-get clean - this removes cached packages, to free up disk space

$ sudo apt-get autoclean - this deletes partially downloaded packages and various bits of computer detritus

$ sudo apt-get -f install - this does a check for packages which might have broken whilst you tried to install them

This next bit takes much longer - it checks for 'bad blocks' on your hard drive - its like the 'disk defragmenter' tool in newer versions of Windows. Basically, it checks your disk (mine is /dev/sda1 but yours might be something else - run 'fdisk' to find out) for physical defects

$ sudo badblocks -v /dev/sda1

Now I don't know about you but I have some software which crashes a lot, or just doesn't seem to work at all. Run these commands if you're worried about malicious software or corrupt pacakges

$ sudo aptitude install debsums - this will install the necessary package 'debsums'

$ debsums - this will run the check (warning - if your computer is as full as mine, this will also take a very long time - allow an hour or so)

Linux doesn't suffer heavily from internet viruses and malware/spyware, but it is known for even linux machines to become infected and it is becoming more common. So here's how to install a firewall (on ubuntu):

$ sudo apt-get install firestarter - firestarter is a fancy interface to the more general linux 'iptables' program. To see whether your firewall is configured type sudo iptables -L into a shell. If the output doesn't have a list of rows starting with 'ACCEPT', it is not configured.

Then go to System>Administration>Firestarter - follow the options easy peasy! Make sure when it first starts to select the option, under 'Preferences', 'minimise to tray on close'

The firewall will run in the background and monitor and do what it does when you're surfing the web. To activate the firewall, type 'firestarter' into a shell before you browse

Ok - cleaned, defragmented, firewalled. And drought over! Man, that was rather professional!