SysadmandineOne day, sysadmins will rule the world…

Sometimes, you need more space on a virtual machine disk than you thought in the beginning. Hopefully, if your domU disk is in a .img file, you can do it quite easily (you can do it easily with lvm partition too, even if it’s a different method). Here’s how to do it, as root of course :

- Check your partitions in your domU :

- create empty file of the size we want to add : (10gb here)

- stop your domU

- backup img files

- add the empty file to the img file :

You can repeat this step as many times as you need

- tell the filesystem to check and adapt its size :

- restart your domU

- check in your domU that everything is ok

Voila! Your disk is now bigger :)

You can also use this method to create a brand new blank disk image, and add it as a new disk for you domU :

or

add the new disk (xvda3) to your /etc/xen/mydomU.cfg :

after rebooting you domU, check if the new disk is available :

You’re now free to create your filesystem on this disk, and to mount it wherever you want!

This Howto was made under Ubuntu Hardy, Xen 3.2

I’m trying to develop a new website to increase my php object oriented skills. For this new website, I want every request for any url that doesn’t match a actual file on the disk to be redirected to index.php (to handle parameters in fact). Easy with apache2 rewrite rules :

This means : if the requested file is not a real file, and isn’t a directory, and isn’t a symlink, then redirect to index.php.

I was really surprised to discover that it doesn’t work. Though, everybody seems to use this syntax ! I checked my apache version : Apache/2.2.9 (Debian), nothing special with this one I guess.
To understand what Apache was doing with my rewrites, I activated the rewrite log :

Here’s what I got (the interesting part, cause I got a looot more !) :

So apaches verifies only ‘/toto.htm’ and not the whole path for “%{REQUEST_FILENAME}”? I thought though it was the whole path… let’s verify in the doc.
From http://httpd.apache.org/docs/2.0/mod/mod_rewrite.html, by habit (cause I used apache 2.0 a lot more than apache 2.2 from now on) :

REQUEST_FILENAME : The full local filesystem path to the file or script matching the request.

Hmm. But I use apache version 2.2, so what do they say here http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html :

REQUEST_FILENAME : The full local filesystem path to the file or script matching the request, if this has already been determined by the server at the time REQUEST_FILENAME is referenced. Otherwise, such as when used in virtual host context, the same value as REQUEST_URI.

Ow.

REQUEST_URI : The resource requested in the HTTP request line. (In the example above, this would be “/index.html”.)

Ok, I understand, I use virtual hosts (like everybody, uh?), so the real syntax for my needs is :

This works even if it doubles the “/” between each variable (one / at the end of DOCUMENT_ROOT, and another at the beginning of REQUEST_FILENAME).

Here’s the rewrite log showing that it works :

Now I can disable this log if I want to keep space on my disk.

I must admit I read the description for REQUEST_FILENAME in apache2.2 several times before noticing that it was just the answer… too used to read too fast! Thanks to this old post that made me re-read slower ! ;)

Here’s my linux favourite commands, by type :

Scripting
1. egrep : aaah… if I had to keep only one, this would be the one I’d choose… can’t spend 10 minutes without using it.
2. sed : because I always need to replace something by something else 10 times on each line in huge files
3. xargs : helps me avoid the for loop, shorter to write, easier to remember and understand when you find it in history
4. awk : for me the most difficult one to use, but I so know that it can do just what I want that I always take the time.

System
5. kill : ha ha, makes me feel I’m powerful
6. ps fauxww : makes me think I can watch everything that’s happening in real time. Like watching 10 movies at the same time.
7. aptitude : I like his way of telling me everything of what he’s doing… so chatty !
8. dstat : this one is really good : tells me a lot of things and with colors, I think he just knows what I’m looking for each time I call him.

Network
9. ssh : is it necessary to say something about it? without him I’d be nothing.
10. telnet : I like the raw output, no interpretation, he’s stupid and this is sometimes really appreciable
11. host : simple tool for so great usefulness, even the basic usage is perfect
12. nmap : I looove knowing everything about hosts I can’t access…
13. tcpdump : really useful when you really dont understand what’s happening.
14. netstat : nmap from the inside. Feels like a treasure hunt sometimes.
15. mtr : seems just easy and fast, but also gives a lot of interesting information.

Tools
16. vim : makes me save so much time
17. screen : How did they live before screen? And I use only a small amount of its potential…

Storage
18. fdisk : boring in the begenning, fun once you know it by heart
19. sfdisk : really handy to mirror a disk partitionning scheme
20. mdadm : I love mdadm. Just love it, don’t know why.

I was wondering how crypt does the encryption of passwords. Once I needed to compare two .htpasswd files, I was quite sure that both had the same password for the same user, but the encrypted strings didn’t match. I tried to regenerate the encrypted string several times, and here’s what I got :

So, how does it work? How is it possible to compare a password with its encrypted version when you can’t be sure to have the same encryption string each time you encrypt it?
I googled and found that it has something to do with the “salt”. the salt seems to be the 2 first letters of the string, and the password is generated considering those. To be sure, I generated the same password enough times to see :

After looking closely to the sorted file, I can say that every time the first 2 letters are identical, the second part of the encrypted string is also identical.

To try to generate a password with a specified salt, you can use the perl crypt function :

And then you can see that this comand outputs exactly the same string as the htpasswd had generated above !
Mystery solved ! ;)

I often need to execute a loop x times, and I’m just too lazy to write a i++ style algorithm… the command seq is made for me :)

Easy to use, just does what I need from it, here’s a extract from the man page :

So…

Great ! :p

Pound is a wonderful tool. It does just what you want, and does it good. Well, most of the time actually. Let me present it.

Pound is made by a Swiss company named Apsis GmbH (http://www.apsis.ch/), and its purpose is to achieve 3 goals :
- reverse proxy
- load balancing
- HTTPS front-end

You don’t need to use all 3 features, but you might need it anyway. Don’t assume it can do other things : No caching, no webserver, no acceleration. Just what it’s meant for.

Read the rest of this entry »

I have this very special server at work : a sunfire x4200 8 core 8GB RAM
To use the full given power, I tried to install xen on ubuntu, but it were freezing all the time. It seems to be a “TAP” problem, but I couldn’t fix it.
So I tried a Debian 5.0.3 dom0, and the problem is slightly different : now the domU won’t find its drives if I use “tap:aio”.
Ok, go back to the old “file:” driver.
This one is using loop devices to enable the drives, and only 8 default loop devices are enabled on debian (/dev/loop0 to /dev/loop7). As I want 7 domU, and at least 2 drives for each one (root and swap), I need more than 8 of these.
Here’s how to add more loop devices :
On Debian and ubuntu loop is a module, and you can specify the number of loop devices you want in /etc/modules. Replace :

by

(64 for example)

Then you have to disable the module :

and re-enable it :

But the new devices won’t append automatically for sure, so if you don’t see them in /dev/loop*, you can make them appear this way :
Edit /sbin/MAKEDEV and change :

By:

And execute

For me this step didn’t work either, so I had to create it really by hand :

My new HDD has just arrived ! 1 more terabyte to play with :D
Now the funny part begins : here’s what I already have in my computer :
- 1 x 500GB for all my operating systems
- 2 x 1TB RAID1 for my documents

My documents partition is full, so I want to switch from 2 x 1TB RAID1 to 3x 1TB RAID5.
I thought it would be easy, as I could read here and there, that 4 command could do the trick :

- stop the RAID1 /dev/md0 array :

- create the raid5 with only 2 disks in place of the old raid1 :

- add the third HDD :

- and use all the available space :

But ! Because it wouldn’t have been fun if it were too easy, the first step didn’t work :

Anything I could try made this work (–force, recovery root shell, runlevel 1, degrade the array, kill all my processes…).

The reason, I guess (no, I’m sure), is that I created a lvm partition on that raid array, and this is what prevents me from stopping the array. So considering this, I wanted to try something different, this was the plan :
1) degrade the raid1 array so keep the data on only 1 disk
2) create a raid5 array on the 2 other disks
3) move my lvm partition on that new array
4) delete the lvm physical volume from the old raid1
5) stop (finally) the old array
6) add the last disk to the raid5
7) grow everything to fit (physical volume, logical volume and filesystem which is reiserfs)

Let’s give it a try
1) degrade the raid1 array so keep the data on only 1 disk

2) create a raid5 array on the 2 other disks

3) move my lvm partition on that new array
The hard part !
3.1) initialize the partition on the raid5 array for use by LVM

3.2) add the new physical volume to the docs volume group

3.3) Move the data from md0 to md1 (This step takes a long time.)

3.4) Remove md0 physical volume from volume group

4) delete the lvm physical volume from the old raid1

5) stop (finally) the old array

And this is where I know I was right from the beginning for doing all this ;)

6) add the last disk to the raid5

And don’t forget to tell mdadm to actually use it (not spare it) =)

This step takes a loooong time too.

7) grow everything to fit (physical volume, logical volume and filesystem which is reiserfs)
physical volume :

logical volume :

filesystem :

Done ! :)
And everything has been made online on a mounted partition ! :-)

Oh, and I was about to forget! if I want my new array back next time I reboot :

I have to replace the old line in /etc/mdadm.conf by this one !

For more convenience, I removed the “sudo” in front of most of all the above commands, but of course you have to execute those with root privileges.

Today I struggled with the time command.

I wanted to use it in a bash script, so I checked the man page, where I found interesting stuff :

Ok great. So I wrote my command like this :

hm. Maybe I did something wrong. Let’s try the man page’s example to understand how things work :

wtf??! The example given didn’t work either !
And then

ok… so, searching a little bit further helped me find the explanation :

Your shell (apparently bash) provides a builtin time function. If you want to use time binary, call it by absolute path (/usr/bin/time)

let’s try :

Finally! It was just a silly thing…

Just remember, when using the bash time, get the time in minutes/seconds:

and in seconds only :

:D

Today I learnt a new command :D

Ok I’m a little bit lying : I knew it before but I was using it only to put several lines into one unique line, like this :

Now, thanks to the new guy in front of me (the colorblind one), I will be able to replace a lot of “for” and ` ` by xargs :D

Some interesting uses of xargs :

Display the content af the current directory in 8 colums

Seems to be more efficient than “find /path -exec rm {} \;”, because xargs splits this list into sublists and calls rm once for every sublist, while find calls rm once for every single file.

to get uptime of each server in server.list