SysadmandineOne day, sysadmins will rule the world…

I love apache mod_rewrite. I often fight with it, but if you really want it you can do whatever you want. This time I wanted to find a way to call a file named after the called subdomain. I have a website called www.yellow-sub.net, and there is some subdomains like the-beatles.yellow-sub.net, john-lennon.yellow-sub.net and so on for each part of the website (That’s a SEO consideration in the beginning). I have only one virtual host for that website because I find it easier to manage like that, and I didn’t want to put the logic in the files for some reasons (and because working with rewrite rules is fun).

This is what I want :

• you call -> you get
• www.yellow-sub.net -> index.php?page=somepage
• the-beatles.yellow-sub.net -> index.php?page=somepage-the-beatles
• blahblah.yellow-sub.net -> index.php?page=somepage-blahblah

The page parameter will in fact call a different page with name “somepage-blahblah.html”, and I wanted it to work for every single subdomain (if it doesn’t exist it will anyway be redirected to www home page)

The closest way I found was to use %{HTTP_HOST} in my rewriterule :

But this way I had to name my files somepage-the-beatles.yellow-sub.net.html instead of somepage-the-beatles.html, wich is not so bad but quite annoying when I want to copy that from beta (different domain name) to live or from live to beta.

So I found the Rewritemap parameter in the apache documentation website. And That’s just an amazing new feature I didn’t know about : it will let you do whatever you want with your rewrite rules! I let you read the manual page, It’s well written and I won’t explain something already well explained : See that Rewritemap doc

I used the last MapType : External Rewriting Program that allows you to write your own script in whatever programming language you fancy, taking stdin and writing to stdout. Mine is very simple, in bash :

I guess that’s a good practive to Keep It Simple, Stupid, as advised in the manual, cause it will be running as apache child, not called when needed. You can test it by giving it something to process :

To use it in you Virtualhost section :

Don’t be messy with the interesting part ${subdomain:%{HTTP_HOST}}, I sillily wrote a % instead of the $ and spent half an hour to figure it out! (no errors anywhere, the script was launched, it just didn’t work at all).

Reload apache to watch it running now :

You can watch it in action using the Rewritelog :

And now my files are just called the way I wanted to :)
I have to say, I love this feature more because I can imagine a lot of new functions in my rewriterules than because it solved my problem, for which I had other options anyway (not as sexy ones, but solutions though.)

The question now is : it is safe? What are the risks of using such a script within apache? If you have anything to say about that, please tell me :)

Have fun with mod_rewrite!

I’m sure you heard of the new unofficial domain registry 42registry, who let you register domains like something.42. If you didn’t, go on there website and read, cause I found this very interesting, funny, and it made me want to try :)

So, after registering sysadmandine.42 domain name, I tried to surf on it. But I didn’t want to spend time to configure a secured but open bind on one of my servers, so I opted for the easy way : my local computer is on Ubuntu, Ubuntu is perfectly able to provide me a bind, so let’s go :

Then one line of config like they say on the 42registry website :

add :

restart bind (surprisingly, just reload didn’t do it) :

Check that it’s working :

(you have to see a ANSWER section id it’s working)

And then tell your system to use it :

Put this on the first line :

That’s it ! Now you just have to open your firefox or any other browser (be careful with chrome or chromium though, ther’s a trick to make it work with those ones, check the 42registry website to know how.), and try some domains !
* http://www.sysadmandine.42/ (of course)
* http://nic.42/
* I couldn’t find more websites in .42, do you know some?

Sometimes, you need more space on a virtual machine disk than you thought in the beginning. Hopefully, if your domU disk is in a .img file, you can do it quite easily (you can do it easily with lvm partition too, even if it’s a different method). Here’s how to do it, as root of course :

- Check your partitions in your domU :

- create empty file of the size we want to add : (10gb here)

- stop your domU

- backup img files

- add the empty file to the img file :

You can repeat this step as many times as you need

- tell the filesystem to check and adapt its size :

- restart your domU

- check in your domU that everything is ok

Voila! Your disk is now bigger :)

You can also use this method to create a brand new blank disk image, and add it as a new disk for you domU :

or

add the new disk (xvda3) to your /etc/xen/mydomU.cfg :

after rebooting you domU, check if the new disk is available :

You’re now free to create your filesystem on this disk, and to mount it wherever you want!

This Howto was made under Ubuntu Hardy, Xen 3.2

I’m trying to develop a new website to increase my php object oriented skills. For this new website, I want every request for any url that doesn’t match a actual file on the disk to be redirected to index.php (to handle parameters in fact). Easy with apache2 rewrite rules :

This means : if the requested file is not a real file, and isn’t a directory, and isn’t a symlink, then redirect to index.php.

I was really surprised to discover that it doesn’t work. Though, everybody seems to use this syntax ! I checked my apache version : Apache/2.2.9 (Debian), nothing special with this one I guess.
To understand what Apache was doing with my rewrites, I activated the rewrite log :

Here’s what I got (the interesting part, cause I got a looot more !) :

So apaches verifies only ‘/toto.htm’ and not the whole path for “%{REQUEST_FILENAME}”? I thought though it was the whole path… let’s verify in the doc.
From http://httpd.apache.org/docs/2.0/mod/mod_rewrite.html, by habit (cause I used apache 2.0 a lot more than apache 2.2 from now on) :

REQUEST_FILENAME : The full local filesystem path to the file or script matching the request.

Hmm. But I use apache version 2.2, so what do they say here http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html :

REQUEST_FILENAME : The full local filesystem path to the file or script matching the request, if this has already been determined by the server at the time REQUEST_FILENAME is referenced. Otherwise, such as when used in virtual host context, the same value as REQUEST_URI.

Ow.

REQUEST_URI : The resource requested in the HTTP request line. (In the example above, this would be “/index.html”.)

Ok, I understand, I use virtual hosts (like everybody, uh?), so the real syntax for my needs is :

This works even if it doubles the “/” between each variable (one / at the end of DOCUMENT_ROOT, and another at the beginning of REQUEST_FILENAME).

Here’s the rewrite log showing that it works :

Now I can disable this log if I want to keep space on my disk.

I must admit I read the description for REQUEST_FILENAME in apache2.2 several times before noticing that it was just the answer… too used to read too fast! Thanks to this old post that made me re-read slower ! ;)

Here’s my linux favourite commands, by type :

Scripting
1. egrep : aaah… if I had to keep only one, this would be the one I’d choose… can’t spend 10 minutes without using it.
2. sed : because I always need to replace something by something else 10 times on each line in huge files
3. xargs : helps me avoid the for loop, shorter to write, easier to remember and understand when you find it in history
4. awk : for me the most difficult one to use, but I so know that it can do just what I want that I always take the time.

System
5. kill : ha ha, makes me feel I’m powerful
6. ps fauxww : makes me think I can watch everything that’s happening in real time. Like watching 10 movies at the same time.
7. aptitude : I like his way of telling me everything of what he’s doing… so chatty !
8. dstat : this one is really good : tells me a lot of things and with colors, I think he just knows what I’m looking for each time I call him.

Network
9. ssh : is it necessary to say something about it? without him I’d be nothing.
10. telnet : I like the raw output, no interpretation, he’s stupid and this is sometimes really appreciable
11. host : simple tool for so great usefulness, even the basic usage is perfect
12. nmap : I looove knowing everything about hosts I can’t access…
13. tcpdump : really useful when you really dont understand what’s happening.
14. netstat : nmap from the inside. Feels like a treasure hunt sometimes.
15. mtr : seems just easy and fast, but also gives a lot of interesting information.

Tools
16. vim : makes me save so much time
17. screen : How did they live before screen? And I use only a small amount of its potential…

Storage
18. fdisk : boring in the begenning, fun once you know it by heart
19. sfdisk : really handy to mirror a disk partitionning scheme
20. mdadm : I love mdadm. Just love it, don’t know why.

I was wondering how crypt does the encryption of passwords. Once I needed to compare two .htpasswd files, I was quite sure that both had the same password for the same user, but the encrypted strings didn’t match. I tried to regenerate the encrypted string several times, and here’s what I got :

So, how does it work? How is it possible to compare a password with its encrypted version when you can’t be sure to have the same encryption string each time you encrypt it?
I googled and found that it has something to do with the “salt”. the salt seems to be the 2 first letters of the string, and the password is generated considering those. To be sure, I generated the same password enough times to see :

After looking closely to the sorted file, I can say that every time the first 2 letters are identical, the second part of the encrypted string is also identical.

To try to generate a password with a specified salt, you can use the perl crypt function :

And then you can see that this comand outputs exactly the same string as the htpasswd had generated above !
Mystery solved ! ;)

Pound is a wonderful tool. It does just what you want, and does it good. Well, most of the time actually. Let me present it.

Pound is made by a Swiss company named Apsis GmbH (http://www.apsis.ch/), and its purpose is to achieve 3 goals :
- reverse proxy
- load balancing
- HTTPS front-end

You don’t need to use all 3 features, but you might need it anyway. Don’t assume it can do other things : No caching, no webserver, no acceleration. Just what it’s meant for.

Read the rest of this entry »

I have this very special server at work : a sunfire x4200 8 core 8GB RAM
To use the full given power, I tried to install xen on ubuntu, but it were freezing all the time. It seems to be a “TAP” problem, but I couldn’t fix it.
So I tried a Debian 5.0.3 dom0, and the problem is slightly different : now the domU won’t find its drives if I use “tap:aio”.
Ok, go back to the old “file:” driver.
This one is using loop devices to enable the drives, and only 8 default loop devices are enabled on debian (/dev/loop0 to /dev/loop7). As I want 7 domU, and at least 2 drives for each one (root and swap), I need more than 8 of these.
Here’s how to add more loop devices :
On Debian and ubuntu loop is a module, and you can specify the number of loop devices you want in /etc/modules. Replace :

by

(64 for example)

Then you have to disable the module :

and re-enable it :

But the new devices won’t append automatically for sure, so if you don’t see them in /dev/loop*, you can make them appear this way :
Edit /sbin/MAKEDEV and change :

By:

And execute

For me this step didn’t work either, so I had to create it really by hand :

My new HDD has just arrived ! 1 more terabyte to play with :D
Now the funny part begins : here’s what I already have in my computer :
- 1 x 500GB for all my operating systems
- 2 x 1TB RAID1 for my documents

My documents partition is full, so I want to switch from 2 x 1TB RAID1 to 3x 1TB RAID5.
I thought it would be easy, as I could read here and there, that 4 command could do the trick :

- stop the RAID1 /dev/md0 array :

- create the raid5 with only 2 disks in place of the old raid1 :

- add the third HDD :

- and use all the available space :

But ! Because it wouldn’t have been fun if it were too easy, the first step didn’t work :

Anything I could try made this work (–force, recovery root shell, runlevel 1, degrade the array, kill all my processes…).

The reason, I guess (no, I’m sure), is that I created a lvm partition on that raid array, and this is what prevents me from stopping the array. So considering this, I wanted to try something different, this was the plan :
1) degrade the raid1 array so keep the data on only 1 disk
2) create a raid5 array on the 2 other disks
3) move my lvm partition on that new array
4) delete the lvm physical volume from the old raid1
5) stop (finally) the old array
6) add the last disk to the raid5
7) grow everything to fit (physical volume, logical volume and filesystem which is reiserfs)

Let’s give it a try
1) degrade the raid1 array so keep the data on only 1 disk

2) create a raid5 array on the 2 other disks

3) move my lvm partition on that new array
The hard part !
3.1) initialize the partition on the raid5 array for use by LVM

3.2) add the new physical volume to the docs volume group

3.3) Move the data from md0 to md1 (This step takes a long time.)

3.4) Remove md0 physical volume from volume group

4) delete the lvm physical volume from the old raid1

5) stop (finally) the old array

And this is where I know I was right from the beginning for doing all this ;)

6) add the last disk to the raid5

And don’t forget to tell mdadm to actually use it (not spare it) =)

This step takes a loooong time too.

7) grow everything to fit (physical volume, logical volume and filesystem which is reiserfs)
physical volume :

logical volume :

filesystem :

Done ! :)
And everything has been made online on a mounted partition ! :-)

Oh, and I was about to forget! if I want my new array back next time I reboot :

I have to replace the old line in /etc/mdadm.conf by this one !

For more convenience, I removed the “sudo” in front of most of all the above commands, but of course you have to execute those with root privileges.

I’m currently trying to find a backup program that fits my need. I found backup-manager, which seems to be easy to install, easy to use, and does what I want : filesystem backup in tarballs, incremential backups, even database backups or svn backups.

On ubuntu, just aptitude it :

The config file is here : /etc/backup-manager.conf.
Take the time to fill it correctly, it’s very well explained inside.

If you want different types of backup on the same server, just write several config files, and call backup-manager this way : backup-manager -c /etc/backup-manager-alternate.conf

One last thing : always call it with root privilegies.

Now I’m just wondering how I could easily manage config files and what to backup for all servers… maybe build it dynamically thanks to a database of installed services before sending them on each server.