If you’re running a company mainly occupied with development of lots of web sites ranging from tiny promo sites to large portals you most probably should face deployment headaches some day.

How do folks usually deploy their remote applications? Some use manual deployment via FTP, others prefer using rsync or unison, some brave souls deploy simply by updating the project’s Subversion working copy on the remote server, Rails fans usually stick with Capistrano, some smart fellows are said to be using complicated cfengine, etc.

At my company we were not happy with any mentioned above methods of remote project deployment due to either lack of functionality or unnecessary complexity. We simply needed a tool which could use non-technical personnel: managers, web-designers, etc. That’s why we hacked our own simple utility - meet Syncman

Syncman is a Limb3 based application which simplifies projects remote deployment and synchronization by providing both nice web UI and basic shell interface.

Behind the scene

So, how does it work? It’s pretty straightforward, there are four stages of deployment with Syncman:

1. The project to be deployed is checked out(or updated) from Subversion repository to the temporary directory located on the Syncman server somewhere in your intranet.
2. Project’s presync_cmd command is run on the Syncman server. At this stage project’s file system layout is prepared for deployment: unnecessary development files and directories removed, static files are prepared for efficient serving, etc.
3. Deployment stage. Usually rsync is used over SSH channel with public keys infrastructure as a transport. However you can change this and use whatever transport suits you the best in the certain case. For example, some folks reported to use FTP tools like ftpsync or sitecopy as transport, since some hosters may not provide access to rsync or SSH.
4. Project’s postsync_cmd command is executed on the remote production server. This command is usually run over SSH, however, as with presync_cmd you can change this. You can try triggering this command remotely with curl via HTTP, or use FTP SITE feature if provider doesn’t provide SSH access. At any rate, this command is used to cleanup any stale caches and other files which became obsolete after project deployment.

Some screenshots of Syncman at work:

• Projects overview(green color - project is in sync, red - out of sync)
  

• Viewing project’s diff
  

• Deploying a project
  

Putting project under Syncman’s control

You need to follow these steps to add the project under Syncman’s control:

1. Install Syncman on some server located in your Intranet
2. Bootstrap the project on the remote host: create a remote folder, add a new virtual host, set up a database, etc
3. Add new project to Syncman directory and configure project’s setting.ini file(see below for example)
4. That’s it, from this moment non-technical personnel can deploy projects using Syncman’s web interface. Alternatively it’s possible to start deployment process using sync.php shell script located in Syncman’s bin directory.

The step 1 is, obviously, done only once, steps 2-3 are usually done once for the project while the last step 4 is repeated quite often.

Ok, let’s try putting some imaginary project under Syncman’s control(provided steps 1-2 are done already). Say, our project is called my.com and it’s available in our intranet Subversion repository.

1. Log in via SSH on to the Syncman server
2. Go to the Syncman directory
3. Proceed to the projects folder and create my.com directory in it
4. Create settings.ini with the following contents in my.com project directory:

   host=myisp.com
   user=syncman
   key=/home/syncman/.ssh/id_dsa
   repository=svn://localhost/my.com/trunk
   remote_dir=/var/www/my.com
   presync_cmd=php %local_dir%/cli/pre_sync.php
   postsync_cmd=ssh -i %key% %user%@%host% 'php %remote_dir%/cli/post_sync.php'
   

   In this case we are assuming that rsync over SSH with public keys synchronization is used. But, as mentioned above, you can try some other sync_cmd, for example:

   sync_cmd=ftpsync.pl -n -P -v %local_dir% ftp://%user%:%key%@%host%/%remote_dir%
   

   As you may have already noted, you can use any settings variables using %var_name% syntax.

   Below we create scripts defined as presync_cmd and postsync_cmd options.

5. Create cli/pre_sync.php script in your project’s working copy and commit it:

   <?php
   $dir = dirname(__FILE__);
   echo "Pre syncing...";
   `rm -rf $dir/../tests`;
   echo "done.n";
   ?>
   

   In this script project’s tests directory is removed since it’s not needed on the production server. Remember, this script is run locally before deployment procedure.

6. Similarly create and commit cli/post_sync.php script in your project’s working directory. This script is executed remotely after deployment procedure:

   <?php
   $dir = dirname(__FILE__);
   echo "Post syncing...";
   `rm -rf $dir/../var/cache`;
   `rm -f $dir/../var/db_info*`;
   echo "done.n";
   ?>
   

   In the script above cache files removed on the remote production server.

Sync it!

Now you have everything ready for deployment. Open web address where your Syncman server is located. For example, in my company it’s a syncman.bit host. If everything was done properly you should see your my.com project listed.

Try deploying this project by clicking the ‘Sync’ icon: a new window should pop up showing deployment log in detail.

Syncman’s TODO

Of course Syncman is far from perfect and here’s what it is currently missing badly:

1. Support for Capistrano alike deployed projects layout: the new version of the project is deployed to the separate folder and the process is complete, the “current” project symlink is simply pointed to this folder. This allows to have a history of recently deployed projects and thus it makes possible to rollback quickly if something went wrong. Another neat feature of such file system layout is atomic deployment.
2. More responsive UI for many projects. We at my company have more than a hundred of projects deployed with Syncman and it takes about 10 seconds for the main page to refresh since each time the page is reloaded svn status is called for each project. Definitely a very ineffective design solution.
3. User authentication and per-project permissions. Well, you can use external Apache http authentication as a protection mechanism(to some extent), however at some moment a more fine grained access control is obviously needed.
4. Some sort of multiple tasks support, somewhat like in Capistrano but a lot more simple(remember, Syncman is simple). The idea is to specify some arbitrary flags per project which would be passed as environment variables to pre- and post-sync scripts. This should allow to have conditional logic in these scripts which may come quite handy at times.
5. Multiple servers deployment support. Currently Syncman can be used for very simple deployment model where only one server is used and this server plays several roles at the same time: an application server, a web server and a database server.

Anyway Syncman does its job pretty good and I hope it can be useful for you as well. At the moment there’s no official release, Syncman is available via Subversion repository only. Please visit http://syncman.limb-project.com for details(there’s also information on how to provide feedback and requests for features). I hope to release official version of Syncman on Sourceforge once this ticket is resolved.

P.S. I have discovered Webistrano, a web UI for managing Capistrano deployments. If you need the power of Capistrano you definitely should have a look at it.

This entry was posted on Sunday, October 7th, 2007 at 6:26 pm and is filed under limb, syncman, *nix, deployment, php. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.