Tag Archives: svn

Accessing a remote svn repository on a non-standard port with Win7

Accessing a remote svn repository that provides SSH on a non-standard port with a Windows7 computer has its own special quirks you have to figure out. Here is how I figured it out.

Windows 7 users, how to use svn

Download & install Tortoise SVN http://tortoisesvn.net/downloads.html TortoiseSVN effectively installs subversion on your system, but you have to make some provisions when the host has installed ssh on a non-standard port, which is done sometimes for security reasons. So for this reason, we will use tortoisesvn for svn but you wont be interacting with it for this website. TortoiseSVN seems to have some kind of issue with connecting to non-standard ports for ssh, and this is really the problem that brought about this little exercise in the first place. So you install TortoiseSVN, but you will use a command line tool to work with subversion for this workaround, and this is explained further down the post. Next bit of housekeeping is specifying the port.

Specifying a port

  • We have to specify a port for the web host, because ssh is on a non-standard port, 10222, instead of 22.
  • In your home directory, locate the .ssh directory. You may need to change your folder settings to show hidden files.
  • create a file in this folder called config not config.txt, just config.
  • open and edit config
  • add the following lines and save:
    Host abcdefdomain.ca
    User abcdefuser
    Port 10222
  • save and exit that shit.

Dealing with a non-standard port

Because of the non-standard port, you will need to use git-bash
from the command line, using git-bash http://code.google.com/p/msysgit/
This software installs git on your system, but actually is a shell environment much like terminal in mac, or puttySSH, which allows you to navigate a windows system with bash commands like ls, pwd, rm, ping, cd, mkdir, and others. Also installs Vim. This app is very handy for us shell zealots when we have our hands on a windoze machien that we keep around for MS Project and Steam games.

Make a directory either in windows or git-bash. Use git-bash to get into that directory.
The command might look like this:

cd ~/websites/codebase/

Then from the command line, you enter the following command:

svn checkout svn+ssh://username@xyzabcdomain.com/home2/path/to/repository/dev3_repo/repo/trunk .

This command is all in one line. Note the local dot at the end! You wouldnt add that using TortoiseSVN.
You will be prompted twice for the password. Once for svn, once for ssh.
A big dump of text should follow on the screen if it was done correctly.

Checking in changes:

if you have made changes to a local file and wish to commit it to the repository, go to the command line with git-bash where the file is and enter the following command at the command line:

svn commit -m “MB line 43-44, changes to formatting xyz.php file” xyz.php

(all in one line).
You commit the file to the repository, with a message, in quotes, you add your initials so a reader can know it is you, you specify as well as possible the change you made, with line numbers, mention the file name in the message, and then the file name itself after the quotes.
hit return, and enter the password when prompted.

If you want to pull down any changes that have been made recently, go to the directory with the command line and do this:

svn up

OR

svn update

enter the password when prompted.

This post was created because dealing with a remote svn repository on a non-standard port using a windows7 machine has its own special, special problems.

But you are a Git user…. What about GIT ?

This blog post provides a workaround for a problem accessing a source over SSH when it is on a non standard port. I wouldnt be surprised if other clients have similar problems when attempting to connect remotely using a win7 desktop.

Acknowledgement goes out to:

Rick at http://conxentric.com/blog/ for this blog post comment:

http://www.techper.net/2009/01/11/changing-port-number-of-svnssh-subversion-protocol/#12

grepnosvn.sh

A simple tool for grepping without seeing all those svn files:

#!/bin/bash
# run this script with chmod 755 permissions.
# modified grep script
#

grep $1 $2 * | grep -v “\.svn” | less

svn

svn
make svn code directory, call it codebase. Put a trunk, branches, and tags directory below it.
Like so: mkdir -p tif_svn/{branches,tags,trunk}
Import all that codebase under trunk. That means, the whole site. Also, make a directory for your db, and put a db dump in there. If you are running a cron, make a directory called cron, put your cron scripts in there and a text file copy of your crontab. Don’t store passwords in subversion, as in the top level file that your site uses. Make a version of the file without the passwords, call it something like config.orig.php or whatever, and check that in instead. Also, you may need the equivalent of the CVS ignore command called svn propset and make use of it.
#svnadmin create repository_directory

#svn import codebase file:///home/user/pathtorepositorydir/repository_directory -m “initial import”
check out your remote repository into your local machine at the command line:
#svn checkout svn+ssh://user@domain.com/home/user/repository_directory/trunk .
local:
#svn checkout file:///home/user/pathtorepository/repository_directory html/

now you are checked in and out, you can delete codebase directory.

svn export works in a similar way. It pulls out the files from the repository sans .svn directories.
#svn export svn+ssh://user@domain.com/home/user/repository_directory/trunk

Check out the Acquia build of Drupal, from their svn repository:

svn export –force https://svn.acquia.com/drupal/branches/1.x-6.x/  .

Installation Script Using WordPress Subversion Repositories

I’d like to take the example of svn externals from my previous post a bit further. I created a bash script that captures a group of commands that one would need to deploy a vanilla WordPress website with a cherry picking of plugins and themes. This script is a big-ass time saver. It’s especially useful for deployment of websites based on a package like WordPress or Drupal, that host publicly available subversion repositories. I tested it recently and made two websites spending two hours on each site, four hours total.

Web developers as they learn start by downloading packages in zip and tar.gz archives which is fine. A pro technique is to pull in the resource via SubVersion (svn). Svn provides additional features such as svn externals and svn hooks. Svn externals allows for the pulling in of external resources into a defined project in a svn repository. Svn hooks is for additional labor saving scripting that allows for the automation of repeated tasks such as a script that can pass along data from commit messages into other resources in the project such as the bug tracker. Learning these tricks allows for labor savings at every step in the production cycle. We can take these practices to the next level with capturing the commands in a bash script and then organise all the features of a website in a deployment script, something that is done all the time at software majors.

This script is a big-ass time saver.

I took the time to sort this out recently because I had some web development requests from people who have simple websites that could be converted to WordPress easily enough, but I wasn’t interested in going fishing for plugins and themes twice, and again when the next people come along needing the same thing. The job of a software developer should be to automate processes. I say often that the credo of developers is not to work for a living, but to eliminate work. But this idea is not always employed by developers in all the places it could, and its sometimes even a harder to get a client on board to make full use of methods and procedures that automate tasks and eliminate work.

…the credo of (software) developers is not to work for a living, but to eliminate work.

Lets at least cover the benefits in point form:

  • The script takes a couple of minutes and you save hours.
  • You dont repeat the labor, but you can repeat the use of the script, deploying anywhere else.
  • All your stuff is there at the beginning. Useful for planning, development, and policy across teams.
  • You are organised, and you can develop variations. Svn export may be good enough for your needs.

In pseudo code, here is what your script will do:

  • Set up your repository.
  • Make your directory structure for your project.
  • Check out your repository.
  • Run procedures for svn externals for core WordPress.
  • Run procedures for svn externals for plugins, iterating through data in an external file resource.
  • Run procedures for svn externals for themes, iterating through data in an external file resource.
  • Grab additional resources in an array, iterate through them using wget command, extract them.
  • Cleanup.
  • Commit message.
  • … anything else you can think to do.

The bash file, save as getallwpsvn.sh:

#!/bin/bash
# run this script with chmod 755 permissions.

workPath=$(pwd)

rm -rf filerepository repository www *.zip # this line cleans dir for testing, comment out when done

svnadmin create repository

mkdir -p filerepository/{branches,tags,trunk/{html,db,cron,scripts,themes,plugins,project,selenium}}
# got anything to import into those directories under trunk?
# import into the directories under trunk now
# before the next step
svn import filerepository file://$workPath/repository -m "initial import using getallwpsvn.sh script"
rm -rf filerepository
svn checkout file://$workPath/repository/trunk www
cd www
svn rm html
svn commit -m "rm html temporarily for clean propset"
svn propset svn:externals 'html http://core.svn.wordpress.org/trunk/' .
svn up
cd html/wp-content/
# get plugins from repository http://svn.wp-plugins.org/
# plugins listed in svn.plugins.externals
svn propset svn:externals -F ../../../svn.plugins.externals plugins/
#svn commit "plugins propset" # no commit if no local repository
svn up
# themes repository: http://svn.wp-themes.org/
# themes repository is a bit of a ghost town, none grabbed here
# browse the site and get the zip
# themes listed in svn.themes.externals file, if there are any
svn propset svn:externals -F svn.themes.externals plugins/
svn up

cd themes
# load up on themes
#more human readable format for array

THEMESITES[0]=http://dev.digitalnature.ro/fusion/fusion-wordpress.zip
THEMESITES[1]=http://ericulous.com/?load=googlechrome.zip
THEMESITES[2]=http://ericulous.com/?load=internetcenter.zip
THEMESITES[3]=http://ericulous.com/?load=redbusiness.zip
THEMESITES[4]=http://wordpress.org/extend/themes/download/elegant-box.4.1.1.zip
THEMESITES[5]=http://wordpress.org/extend/themes/download/thirtyseventyeight.4.0.zip
THEMESITES[6]=http://wordpress.org/extend/themes/download/thirtyseventyeight.4.0.zip
THEMESITES[7]=http://wordpress.org/extend/themes/download/constructor.0.6.4.zip
THEMESITES[8]=http://wordpress.org/extend/themes/download/jq.2.4.zip
THEMESITES[9]=http://wordpress.org/extend/themes/download/ahimsa.3.0.zip
THEMESITES[10]=http://wordpress.org/extend/themes/download/retromania.1.3.zip
THEMESITES[11]=http://wordpress.org/extend/themes/download/skinbu.1.0.3.zip
THEMESITES[12]=http://wordpress.org/extend/themes/download/mystique.1.16.zip
THEMESITES[13]=http://wordpress.org/extend/themes/download/lightword.1.9.3.zip
THEMESITES[14]=http://wordpress.org/extend/themes/download/monochrome.2.3.zip
THEMESITES[15]=http://wordpress.org/extend/themes/download/thematic.0.9.5.1.zip
THEMESITES[16]=http://wordpress.org/extend/themes/download/hybrid.0.6.1.zip
THEMESITES[17]=http://wordpress.org/extend/themes/download/new-york.1.0.1.zip
THEMESITES[18]=http://wordpress.org/extend/themes/download/f8-lite.1.3.zip
THEMESITES[19]=http://wordpress.org/extend/themes/download/simplex.1.3.1.zip
THEMESITES[20]=http://wordpress.org/extend/themes/download/cleanr.0.1.2.zip

for s in ${THEMESITES[@]}
do wget "$s"
done

FILES="*.zip"
for f in "$FILES"
do unzip "$f"
done

rm *.zip
rm *.zip.*
cd ../../../
svn commit -m "load in of plugins and themes complete"

cd $workPath
cp $workPath/www/html/wp-config-sample.php  $workPath/www/html/wp-config.php
chmod 777 $workPath/www/html/wp-config.php
chmod 777 $workPath/www/html/wp-content #temporarily, for cache
mkdir $workPath/www/html/wp-content/uploads && chmod 777 $_
touch $workPath/www/html/.htaccess && chmod 777 $_

# do any post processing, other importing now, and commit it if you did.

Set the file permission to chmod 755, and run it from the shell command line as in ./getallwpsvn.sh.

The file you save as svn.plugins.externals:

all-in-one-seo-pack http://svn.wp-plugins.org/all-in-one-seo-pack/trunk
advertising-manager http://svn.wp-plugins.org/advertising-manager/trunk
cforms http://svn.wp-plugins.org/cforms/trunk
google-sitemap-generator http://svn.wp-plugins.org/google-sitemap-generator/trunk
sociable http://svn.wp-plugins.org/sociable/trunk
stats  http://svn.wp-plugins.org/stats/trunk
ultimate-google-analytics http://svn.wp-plugins.org/ultimate-google-analytics/trunk
vipers-video-quicktags http://svn.wp-plugins.org/vipers-video-quicktags/trunk
wordbook http://svn.wp-plugins.org/wordbook/trunk
wp-flickr http://svn.wp-plugins.org/wp-flickr/trunk
wp-super-cache http://svn.wp-plugins.org/wp-super-cache/trunk

The svn.plugins.externals file is a name – resource listing, one per line, when you have more than one resource to define with svn externals.

Please note that you may not need all of this; comment whatever out you want. You dont need to create a local repository, that is only if you are doing team development, or perhaps custom development on themes and plugins. I found though that it was necessary with svn propset directives to create a top-level directory structure wherein is stored all the different directories. The point of this exercise is a pull-in of public resources in a step that you can repeat automatically over and over. It also need not be a very sophisticated script to get the benefits from it.

The Power of svn externals

I have a list of svn tricks over on a previous post, (my) Essential Cheat Sheet of Shell Commands. But I was listening to a very lengthy, and I mean really lengthy Zend podcast, over two hours in length: The ZendCon Sessions Episode 26: Best Practices of PHP Development, with Matthew Weier O’Phinney and Mike Naberezny. I learned a new trick about subversion I would like to share and note here for the future, svn externals .

I had of course heard of svn externals before but a little explanation really enlightened me. You can multiply your power as a developer with svn externals. Alongside your own project under svn control, you can add in other remote projects from remote svn repositories with the svn propset command and they will naturally remain current as you run updates on your own repository.

You can multiply your power as a developer with svn externals.

In the struggle for project housekeeping it can be a chore to keep libraries, plugins, middleware, and other goodies up to date. In general we as developers have a mindset about keeping a project managed by version control, but its an idea that I have seen stopping at the project in question. The project is under svn, the rest of the libraries are from static resources like tarballs. By just extending the power of version control just a little bit further, we have a big labor saver, and we are opened up to the great universe of software. Svn externals gives us mighty lever, because we then have the power to keep in step with all of the other bits and pieces that go along with a project.

Your main project is under version control, but your rich html editor, TinyMCE, is not. Well it can be. Lets use it as an example.

test$ mkdir tmc
test$ cd $_
test/tmc$ mkdir html
test/tmc$ mkdir project
test/tmc$ cd $_
test/tmc/project$ mkdir branches tags trunk
test/tmc/project$ cd -
/home/pbg/websites/test/tmc
test/tmc$ cd html/
test/tmc/html$ ls
test/tmc/html$ emacs index.php
test/tmc/html$ cd ../
test/tmc$ cp -rf html project/trunk/
test/tmc$ ls
html  project
test/tmc$ ls project/trunk/
html
test/tmc$ ls project/trunk/html/
index.php  tiny
test/tmc$ ls
html  project
test/tmc$ rm -rf html
test/tmc$ ls
project
test/tmc$ svnadmin create tmcrepository
test/tmc$ svn import project file:///home/pbg/websites/test/tmc/tmcrepository -m "initial import"
Adding         project/trunk
Adding         project/trunk/html
Adding         project/trunk/html/index.php
Adding         project/branches
Adding         project/tags
Committed revision 1.
test/tmc$ ls
project  tmcrepository
test/tmc$ svn checkout file:///home/pbg/websites/test/tmc/tmcrepository/trunk .
A    html
A    html/tiny
test/tmc$
/test/tmc/html$ svn propset svn:externals 'tinymce https://tinymce.svn.sourceforge.net/svnroot/tinymce/tinymce/trunk' .
/test/tmc/html$    svn commit -m "propset"
/test/tmc/html$    svn up
/test/tmc/html$

So that is how its done from bash with a vanilla website and one repository checked in for your pleasure. Tips to know include wrapping the directory and resource in quotes, don’t create the directory, specify it in propset and let svn create it for you. Go to a directory somewhere else on your system and test your checkout. In the example above the remote repository trunk is checked out. However, you are also able to check out specific branches or even specific revisions if you want the bias more to stability over new features. You can also specify multiple remote repositories by creating a text file with directory and remote resource pairs and point svn propset at that file. Be prepared to handle things that you flub up using svn propedit. Your repository is not broken, but you may have to know how to fix a thing or two that you didn’t do the first time. That is why doing a vanilla procedure like what is described above helps show what the right way is.

So Imagine having a repository somewhere with all the tools you like to work with as part of your own best practices, sitting there, in one place ready to go with one checkout. That whole kit and kaboodle becomes your blank slate from where to start from, but you are already miles ahead of the competition because you already have tabs on all the resources you are going to use in your project. Having it all on hand saves labor and helps foster better practices as a developer.

  • Zend Framework, CakePHP, Symphony, PEAR or whatever middleware turns your crank
  • PHP Unit, or SimpleUnit, or some other unit testing suite
  • WordPress, Drupal, Joomla, or of course any of the related themes and plugins
  • Tiny MCE
  • Jquery, Scriptaculous, Dojo, whatever floats your boat with javascript
  • XDebug,
  • phpmyadmin
  • Integration testing software like Selenium or Molybdenum
  • bloody well anything from sourceForge, or anything else for that matter public and under svn control.

Links for this blog post:

http://www.superwebdeveloper.com/2008/10/08/essential-cheat-sheet-of-shell-commands/

http://devzone.zend.com/article/9930-The-ZendCon-Sessions-Episode-26-Best-Practices-of-PHP-Development

http://beerpla.net/2009/06/20/how-to-properly-set-svn-svnexternals-property-in-svn-command-line/

http://blogs.gnome.org/johannes/2008/02/20/svnexternals-for-noobs/