Category Archives: Web Development

Hypervisor Troubleshooting for Linux Web Development Servers

There are a few things that can cause problems accessing your virtual machine fully from your host machine. Lots of cases work straight away, but some may come about from mucking with settings your first time out, or from importing the virtual appliance from another machine. The following are the key sources for the causes of connectivity problems.

  • Router
  • Hypervisor
  • Virtual Machine
  • Hosts file

Here are some troubleshooting notes that cover most of the problems that I have encountered since I have learned how to set up virtual machines for website development. The following are still loose notes cleaned up from my own local notes file.

Router

Don’t be above rebooting your router if you have a problem. See if it works after that on the same settings you are on, especially if the virtual machine and host interaction works as you expect it to work. If you are in a web cafe, and you cant get it to work, use your mobile phone as a wireless access point. It will likely be more stable than most web cafes out there.

In the hypervisor

Moving a vm over to a new host machine? Turn off appliance, turn off network>enable network adapter, exit virtual box, restart virtual box, re-enable same adapter, boot up vm.

When moving over to my macbook pro, I had to change the name of my bridged adapter to en1. ( how would I find that out?) I also had to change Advanced > Adapter type to Intel PRO/1000 MT Desktop (82540EM). So be aware of your adapter type.

How I usually operate is this way:

I have an internet connection, even if it is only provided by my mobile phone. Mobile phones are very handy for this purpose.

With your virtual appliance turned OFF:

  • Select the virtual machine > Settings > Network > adapter 1
  • If this is a virtual appliance you have
  • Check enable network adapter. Toggle and close and reopen settings if this is an imported VM.
  • Attached to: Bridged Adapter.
  • Name: Your Ethernet connection, basically. On my Mac it says en1: Wi-Fi (AirPort)
  • Advanced: Cable Connected is CHECKED.

In the Linux virtual machine

Find eth0. I want to see my IP addy in my ifconfig like I do in ipconfig in my win7 host.

Use
ifconfig or  ifconfig –a

to see what is up in the terminal window.

Ideally you should be seeing eth0 and a normal looking IP. Something like 192.168.1.80.

Sometimes, eth0 disappears. Are you on NAT? Try bridged.

Try restarting the network interface. Release dhcp…

  • sudo ifconfig eth0 down
  • sudo ifconfig eth0 up
  • sudo dhclient eth0
  • sudo dhclient –r
  • sudo dhclient

http://www.ubuntugeek.com/how-to-release-and-renew-a-dhcp-ip-address-in-ubuntu-10-04-lucid9-10-karmic9-04jaunty.html

Things to do to get eth0 back – try editing your network rules.

Sudo Vi /etc/udev/rules.d/70-persistent-net.rules

Look for eth0 in the NAME= attribute. If there are others, delete them. If there is eth1 but no eth0, You can overwrite eth1 with eth0, save and reboot the virtual machine.

Note, if you have this problem, it seems to work better to remove any superfluous entries in 70-persisiten-net.rules and overwrite the eth# that is up in your ifconfig –a result, then reboot the virtual machine, go to ifconfig –a, and you will see it up.

So now ifconfig –a tells me that my inet addr is 10.0.2.15. a step forward, but we still aren’t talking to the virtual machine seamlessly like it were a remote server.

Zurmo.org has a tutorial for installing their CRM software with a virtual machine. Consider trying their software to get started with virutalization. http://zurmo.org/wiki/how-to-install-zurmo-virtual-machine

NAT – When using NAT, its like putting the virtual machine behind a virtual router. Just like a router at home. Never mind port forwarding. I am at a new place, and bridged doesn’t seem to work. OK, so try NAT. NAT at least allows me to ping a website on the internet, which mysteriously doesn’t work anymore in Bridged. NAT gives me an ip address on eth0 of 10.0.2.15. Bridged gave me 192.168.1.113. I don’t remember what I did. I rebooted my router.

The normal settings are what works consistently. It does take some practice to get it to work, so don’t give up.

Hosts file

  • General: Any time you move your computer to a different network, or reboot your router, your computer gets assigned a new IP address. You have to keep close tabs on your hosts file so it can be updated. Any line in a hosts file that starts with a  ‘#’ is not executed.
  • Windows: C:\Windows\System32\drivers\etc\hosts – find it, right click on it, and allow permissions for the current user to modify contents of the file. Save a shortcut to where you are going to have easier access on it.
  • Mac: /etc/hosts – Mac users should know the password for their user profile. Open Terminal, and use a command line editor such as vi or emacs to open your hosts file. If you don’t know how to use vim or emacs, go find a tutorial on the web and practice it. You must open the file with the sudo command, and correct password challenge, such as sudo emacs /etc/hosts.
  • Ubuntu Desktop: /etc/hosts  - as above, know your user account password, hoping it has privileges to modify the hosts file with sudo. Get a tutorial on using vi, nano, emacs, or other editor when you get into Terminal so you know what to do when you are there.

There are probably more cases where the connectivity to the virtual machine is lost. I will add more solutions to this problem as I encounter and solve the cases for it.

Bridged and NAT Networking Options in Hypervisors

Using a virtual machine as a dev web server is far superior to xampp, for example, because you are able to run the codebase on a real stack, and even possible a mirror of your production stack. You get to use server tools to work on your website, enabling you work capably when time comes to do work on a server.

But what is the difference between NAT and Bridged networking?

NAT creates a VIRTUAL network inside your host in which your virtual boxes live i.e. 10.0.2.x . Bridged creates a network for your virtual machine on the same level as your host.

If you are using NAT networking, then you have created a virtual router between you and your virtual machines – so you would have to open port 80 in order to be able to hit port 80 on your guests.

Bridged networking is like hooking your guests to a SWITCH, and NAT is like hooking your guests to a router …with bridged, you are hooking the guest to the SAME network as the host so, if you use bridged, your guest will get an IP from the router for the given network you are on, and be on the SAME NETWORK as your host and there will be no need to forward ports.

If you use NAT, it created a NEW VIRTUAL NETWORK inside your host, and connects your guest to that.

THE FAT BLOKE has done a far better job of explaining this than I ever will. Go here if you want to know more: https://blogs.oracle.com/fatbloke/entry/networking_in_virtualbox1

http://www.virtualbox.org/manual/ch06.html

https://help.ubuntu.com/10.04/serverguide/network-configuration.html

Thanks to David Turner of http://www.turneris.com/ for explaining this to me.

Virtual Machines for Web Development

  • Use case
  • How to with screenshots
  • Link to trouble shooting on blog.

Use case

If you are a web developer, it is your responsibility to have a development environment to build your sites before building them on the server. We get away with all sorts of things for years that looking back would be called insufficient. As we improve and get more serious about our work we need better systems. I am a small independent developer; I use Mac, Win, and Ubuntu desktop environments interchangeably. I want a system that is common everywhere I go. I want it to be as close to a real web server as possible. I want to set up quickly and completely. I want to even set up clients and team members with the same environments. My development environment requirements have gone from whatever works, to what works well and right. I started off with installing Windows exes of Apache-PHP-MySQL, and then went on to desktop software such as XAMPP and WAMP and MAMP. But I knew that something was missing from these environments as compared to the real ‘production stack’ that the website runs on. No matter how much time was spent on them, they weren’t the real thing. Why not run a mirror of the real stack? I had heard of it being done, and it took a while to figure out but I have it running. Now that I use virtualization as my web development solution, I will never go back.

Use virtualization to mirror your web stack for development and reap the benefits.

With virtualization, I can have a copy of a web server on my computer, that runs the same way a production web server will run, uses the same packages, gives me access to the shell to work the way I would as if I were online, and groups together the entire stack that the site runs on. Virtual machines running virtual LAMP stack offers all these advantages, and more. I have a system that runs and configures the same on my Win 7 machine, my Ubuntu desktop, and my Macbook. I keep many virtual machines stored, and use them for the case required.

  • Common dev setups across different desktops
  • Common dev setups for dev teams
  • Can function independently from the internet and remote web servers
  • Opportunity to run custom stacks
  • Opportunity to deploy VM to cloud
  • Opportunity to use shell tools instead of desktop stand-ins
  • Opportunity to ditch fake setups
  • Can provide offline copy of site to paying customer
  • A real sandbox for practicing new techniques
  • Can deliver code that will run on the production server with a greater degree of reliability

There are many hypervisors out there, and they all have their best use cases

For the purpose of this demonstration, and for my needs I use Oracle VirtualBox. I use this because it is free to use, and easy to work with; I can set up other users with virtual machines regardless of their skill level or use purpose. Other hypervisors may also fit this bill, to some degree, and I encourage exploration. It is good to know also that there is some interoperability of virtual appliances between different brands of hypervisors, or there are methods and workarounds to make it happen.

Hypervisors: http://en.wikipedia.org/wiki/Hypervisor

How to with screenshots

Skill level: Ability to use Desktop and command line environments. Ability to use the software listed below.

Software:

Desktop: Windows, Mac, Ubuntu Desktop.

Oracle VirtualBox https://www.virtualbox.org/wiki/Downloads .

Oracle VM VirtualBox Extension Pack https://www.virtualbox.org/wiki/Downloads .

A text editor, or Komodo Edit http://www.activestate.com/komodo-edit .

Download a Linux Distribution. Consider Ubuntu Server iso http://www.ubuntu.com/download/server . You can also find prebuilt virtual appliances from Bitnami http://bitnami.com/ , Turnkey Linux http://www.turnkeylinux.org/ , if you are ready for something more prebuilt.

Any good web browser (Firefox, Opera, Chrome, Safari, etc).

FileZilla for ftp transfers. https://filezilla-project.org/

PuTTY http://www.chiark.greenend.org.uk/~sgtatham/putty/

Mac, or Ubuntu: Terminal (built in).

Access to your hosts file

Windows: C:\Windows\System32\drivers\etc\hosts The hosts file is a part of computer networking systems. All computers that use networks have a hosts file.

Mac: /etc/hosts

Ubuntu Desktop: /etc/hosts

How to set up a Virtual Machine as a development web server.

  1. Install VirtualBox in your desktop environment.
  2. Install VirtualBox Extension Pack
  3. Open VirtualBox, create a new machine, and give it a few gigs of space.
  4. Start up the ISO you have obtained, just like you were installing Linux on a hard drive.
  5. Log in, run ifconfig to get your inet addr value for eth0.
  6. Find your hosts file on your local machine. Change the file to writeable permissions if needed (Win).
  7. Although you have access to the terminal shell environment from a window in VirtualBox, you can now use that IP address in PuTTY or FileZilla to access the virtual machine.
  8. Enter your IP address in your hosts file, and assign the aliases you want to use for websites.
  9. For setting up Apache2 websites in your virtual machine, refer to tutorial information widely available on the web.
    1. Tutorial 1: http://www.thegeekstuff.com/2011/07/apache-virtual-host/ ,
    2. Tutorial 2: http://www.debian-administration.org/articles/412 ,
    3. Tutorial 3: http://www.linode.com/wiki/index.php/Configure_apache_to_use_virtual_hosts_on_ubuntu_server .
  10. Copy default to your new site name, and edit it as sudo user.
  11. Use sudo a2ensite (sitename) to copy virtual host file in your sites-available folder over to your sites-enabled folder .
  12. You will probably need apache mod_rewrite. Enter sudo a2enmod rewrite at the command line.
  13. Reload server: sudo service apache2 reload .
  14. Once your apache2 virtual hosts have been created, with directories created inside your machine, you can browse to your virtual machine via your web browser. You would be able to go to  http://test if you created a virtual host entry called test, and you had it configured properly.
  15. Get to phpMyAdmin by going to http://(your ip Address)/phpmyadmin .
  16. There are sometimes a few issues to sort out between your hypervisor, your guest machine, and your host. See troubleshooting page.

Now that I use virtualization as my web development solution, I will never go back.

    Connecting up your host machine to your guest virtual machine

    •    Power up VirtualBox, boot your linux virtual machine.
    •    Log in to your virtual machine with the terminal window provided by VirtualBox.
    •    Find and copy your IP address.
    •    The login message should indicate your IP address, but sometimes will be skipped if the load on the machine is too high at the moment of login. This happens fairly regularly.
    •    Use the command ifconfig and look for the address provided by eth0.
    •    You should be using bridged networking in VirtualBox, and the address should look a lot like this: 192.168.1.79 .
    •    Add this address to your hosts file followed by the domains you want to the given address.
    •    Windows Users: you will likely need to change file permissions on your hosts file.
    •    Find the file at C:\Windows\System32\drivers\etc\hosts.
    •    Right click to examine user privileges and make sure that write privileges are enabled for your current User.
    •    The IP address from your virtual machine can change if you reboot your router, change networks, or reboot your host machine. To handle this, create a shortcut or equivalent resource for easy access to the file.
    •    Mac Users: I should know my mac better, but its not obvious how to get to /etc/hosts through the gui. Open Terminal, and enter: sudo (favorite shell editor) /etc/hosts to get at your hosts file.

    Connecting up your host machine to your guest virtual machine

    · Power up VirtualBox, boot your linux virtual machine.

    · Log in to your virtual machine with the terminal window provided by VirtualBox.

    · Find and copy your IP address.

    · The login message should indicate your IP address, but sometimes will be skipped if the load on the machine is too high at the moment of login. This happens fairly regularly.

    · Use ifconfig and look for the address provided by eth0.

    · You should be using bridged networking in VirtualBox, and the address should look a lot like this: 192.168.1.79 .

    · Add this address to your hosts file followed by the domains you want to the given address.

    · Windows Users: you will likely need to change file permissions on your hosts file.

    ·

    · Find the file at C:\Windows\System32\drivers\etc\hosts.

    · Right click to examine user privileges and make sure that write privileges are enabled for your current User.

    · The IP address from your virtual machine can change if you reboot your router, change networks, or reboot your host machine. To handle this, create a shortcut or equivalent resource for easy access to the file.

    · Mac Users: I should know my mac better, but its not obvious how to get to /etc/hosts through the gui. Open Terminal, and enter: sudo (favorite shell editor) /etc/hosts to get at your hosts file.

    Accessing a remote svn repository on a non-standard port with Win7

    Accessing a remote svn repository that provides SSH on a non-standard port with a Windows7 computer has its own special quirks you have to figure out. Here is how I figured it out.

    Windows 7 users, how to use svn

    Download & install Tortoise SVN http://tortoisesvn.net/downloads.html TortoiseSVN effectively installs subversion on your system, but you have to make some provisions when the host has installed ssh on a non-standard port, which is done sometimes for security reasons. So for this reason, we will use tortoisesvn for svn but you wont be interacting with it for this website. TortoiseSVN seems to have some kind of issue with connecting to non-standard ports for ssh, and this is really the problem that brought about this little exercise in the first place. So you install TortoiseSVN, but you will use a command line tool to work with subversion for this workaround, and this is explained further down the post. Next bit of housekeeping is specifying the port.

    Specifying a port

    • We have to specify a port for the web host, because ssh is on a non-standard port, 10222, instead of 22.
    • In your home directory, locate the .ssh directory. You may need to change your folder settings to show hidden files.
    • create a file in this folder called config not config.txt, just config.
    • open and edit config
    • add the following lines and save: 
      Host abcdefdomain.ca
User abcdefuser
Port 10222
    • save and exit that shit.

    Dealing with a non-standard port

    Because of the non-standard port, you will need to use git-bash
    from the command line, using git-bash http://code.google.com/p/msysgit/
    This software installs git on your system, but actually is a shell environment much like terminal in mac, or puttySSH, which allows you to navigate a windows system with bash commands like ls, pwd, rm, ping, cd, mkdir, and others. Also installs Vim. This app is very handy for us shell zealots when we have our hands on a windoze machien that we keep around for MS Project and Steam games.

    Make a directory either in windows or git-bash. Use git-bash to get into that directory.
    The command might look like this:

    cd ~/websites/codebase/

    Then from the command line, you enter the following command:

    svn checkout svn+ssh://username@xyzabcdomain.com/home2/path/to/repository/dev3_repo/repo/trunk .

    This command is all in one line. Note the local dot at the end! You wouldnt add that using TortoiseSVN.
    You will be prompted twice for the password. Once for svn, once for ssh.
    A big dump of text should follow on the screen if it was done correctly.

    Checking in changes:

    if you have made changes to a local file and wish to commit it to the repository, go to the command line with git-bash where the file is and enter the following command at the command line:

    svn commit -m “MB line 43-44, changes to formatting xyz.php file” xyz.php

    (all in one line).
    You commit the file to the repository, with a message, in quotes, you add your initials so a reader can know it is you, you specify as well as possible the change you made, with line numbers, mention the file name in the message, and then the file name itself after the quotes.
    hit return, and enter the password when prompted.

    If you want to pull down any changes that have been made recently, go to the directory with the command line and do this:

    svn up

    OR

    svn update

    enter the password when prompted.

    This post was created because dealing with a remote svn repository on a non-standard port using a windows7 machine has its own special, special problems.

    But you are a Git user…. What about GIT ?

    This blog post provides a workaround for a problem accessing a source over SSH when it is on a non standard port. I wouldnt be surprised if other clients have similar problems when attempting to connect remotely using a win7 desktop.

    Acknowledgement goes out to:

    Rick at http://conxentric.com/blog/ for this blog post comment:

    http://www.techper.net/2009/01/11/changing-port-number-of-svnssh-subversion-protocol/#12