brunotavares.net: ScrapBook

… you really should ckeck out HttpFox : https://addons.mozilla.org/en-US/firefox/addon/6647

It provides all the information that Live HTTP Headers do (and more, like request time tracking ) but with better organization.

At work we had the need to interact with Orange‘s Pikeo service.

Pikeo is a Flickr like photo management/sharing service that has a REST API.

We ended up making a, object oriented, Perl wrapper for the API. The Perl API project is here, will be published to cpan as soon as we finish the documentation.

As always: comments , bugs and feature requests are welcome.

I still use svn for some projects, some of them are python.

Python pre-compiles the python files, so, the first time i run one of this projects i end up with a bunch of .pyc files that i don’t want to add to the repository.

Since svn doesn’t know this files the svn status generates a lot of garbage in the output.

svn has a property ( see svn properties ) called svn:ignore that tells svn to ignore certain files matching a pattern. The problem is that this property is directory/file based so you have to set it, at least, to every directory.

easy way to do this:

$ cd myproject
$ find . -type d | grep -v '.svn' | xargs svn propset svn:ignore "*.pyc"
$ svn commit

I’ve a lot of code that depends on different versions of the same library, its quite hard to maintain the different versions installed in their specific dirs along with all their dependencies without tainting anything else.

To be able to run all of this code in one machine i’ve to make a lot of manipulation to the PYTHONPATH configuration. This is painfull and time-consuming. The plot thickens if i want to test some new version of the same library.

I found out virtualenv, this is a small python utility that bootstraps isolated python environments.

It allows you to maintain your application, libraries and all their dependencies in a isolated environment.

It also installs setup-tools in the virtual environment turning all your new libraries installations into painless tasks.

It comes with a set of handy scripts ( per-environment ) to activate and deactivate the environment so it’s quite simple to switch around environments.

Optionally you can make your environments inherit ( as a fallback ) the system site-packages and, with little effort, your enviromnents can be “relocatable” not being tied to any specific path which makes it possible to move them around your filesystem.

I use svn+apache2 to give access to all my repos. This is acomplished with dav_svn_module that allows you to give DAV SVN acess to one directory on your file system.

I drop all my projects in /servers/svn/rep/ so, for a while, giving DAV access to /servers/svn/rep/ did the trick. The problem arises when you need to give different permissions to each repository ( and i really didn’t want to write one <Directory> directive for each repository ).

Well, fine-grain permissions ( r/w) per repository using DAV can be accomplished using the SVN AUTHZ. You can allow or deny read or write access to one user or user group by repository.

I’ll post a quick walktrough by my setup, it assumes you already have apache2 rolling with dav_svn_module and authz_svn_module.

Apache Configuration

First of all create a vhost/location for your SVN repo:

<VirtualHost *>
  ServerName    svn.yourhost.com

  <Location />
        DAV svn
        # path to the directory holding your projects repositories.
        SVNParentPath /servers/svn/reps

        AuthType Basic
        AuthName "Subversion Repository"
        AuthUserFile /servers/svn/etc/svn-auth-file
        AuthzSVNAccessFile /servers/svn/etc/svn-authz-file

        require valid-user
  </Location>

</VirtualHost>

This above configuration will provide DAV access to /servers/svn/reps, mapping / to /servers/svn/reps. So, if you have a project called app1 you can acess app1 repository by pointing your browser to http://svn.yourhost.com/app1.

Authentication

We autenticate the users using apache builtin basic auth module. Just create the users file /servers/svn/etc/svn-auth-file using htpasswd command line util.
Let’s add three users to illustrate our example: an app1 coder named Ruth, the server admin “bat” and a guest user.
( Well, unfortunatelly the guest user will also have to have a password )

htpasswd -c /servers/svn/etc/svn-auth-file bat
htpasswd /servers/svn/etc/svn-auth-file ruth
htpasswd /servers/svn/etc/svn-auth-file guest

Authorization

Now we need to limit certain users to certain repositories .This is accomplished by adding some rules to the /servers/svn/etc/svn-authz-file file.
First of all state your user groups by adding the following block to the file:

[groups]
admin=bat
others=guest
coders=ruth

As a fail safe, let’s deny read acess to the base dir to anyone except admins

[/]
* =
@admin = rw

Finally, lets give our coder, Ruth, rw acess to our app1 repository.
Since our app1 is open source, let’s also allow our guest to read from the repository to checkout the latest developments.

[app1:/]
@coders = rw
@others = r
@admin = rw

Just restart your apache and checkout app1 by typing:
$ svn co http://svn.yourhost.com/app1/trunk

All over the year’s i’ve made several perl module distributions. I’ve used a diversity of tools to package the modules and create a cpan ready distribution.

I’ve bumped into several issues with the makefiles and the packaging itself but, for sometime now, i’ve a recipe to create a fully cpan compliant distribuition without any manual editing of the non-code distribution files.

First of all create the skeleton of the distro using Module::Starter

module-starter -mb -module=Foo::Bar --author="your name" --email=your@email

This will create your distro structure in the Foo-Bar directory.
Your code will live in the Foo-Bar/lib dir, and the tests in the Foo-Bar/t

Note: the -mb option tells module-starter to use Module::Build instead of ExtUtils::MakeMaker. This materializes in you ending up with a Build.PL script instead of the traditional Makefile.PL. Just remove -mb option if you want a Makefile.PL but ExtUtils::MakeMaker has some quirks that i personaly don’t like so i use Module::Build

Edit Build.PL file in the Foo-Bar directory, it should be similar to this:

#!/usr/bin/perl

use strict;
use warnings;

use Module::Build;

Module::Build->new(
        module_name => 'Foo::Bar',
        license => 'perl',
        requires => {
                'Other::Module' => '0',
        },
        build_requires => {
                'Test::More' => 0,
        },
        create_makefile_pl => 'traditional',
        sign => 0,
)->create_build_script;

The important bits are the ‘requires’ key, wich states all the depencies for you module and the ‘build_requires’ that states extra dependencies to build the module ( including running the tests you ship with the distro )

You can also sign the distro with pgp (which i recommend), just flip the ‘sign’ value to 1.

Now you just have to create the buld script

$ perl Build.PL

and create the first distribution package

$ ./Build dist 

You should have the Foo-Bar-0.01.tar.gz ready to drop on cpan.

Well, did a complete remake on the brunotavares.net site.

I’m now using a completely new set of tools. The design also took a revamp, it’s a simpler, css based, layout which makes it simpler to distribute to all the different pieces of the site.

The apple’s time-machine concept is precisely what i want for my laptop “backups”

For the information sitting in my laptop i don’t want traditional disaster and recovery backups, i want to be able to, quickly, search and restore any version of ( virtually ) any file.

Tthe problem? i don’t own a mac.

The principle of the thing:

Incremental backup of all the files that changed in a short period of time ( let’s say 5 minutes, apple’s time machine does it every hour ).
Create a snapshot of your laptop at the backup time, this implies being able to browse the laptop the exact same way that you browse your current file structure

My current setup:

BACKUPDIR="/backups"
DIRSTOBACKUP="/home /data"
date=`date "+%Y-%m-%dT%H:%M:%S"`
rsync -aP --exclude-from=~/backup.exceptions \
     --link-dest=$BACKUPDIR/current $BACKUPDIR/back-$date $DIRSTOBACKUP

cd $BACKUPDIR; rm -f current ;ln -s back-$date current