Famous Coders Write Blogs

These guys are guns. You should read their blogs. I do.

Eric Lippert – Fabulous Adventures in Coding – Former member of C# language design committee, 16 years working at Microsoft.
Herb Sutter – Sutter’s Mill – God of C++.
Jon Skeet – Jon Skeet’s Coding Blog – Knows C#. Wrote some book. Works for Google.

 

… this is a work in progress!

Virtual Mac OS X Mavericks desktop in the Cloud

I write software for Windows. Exclusively for Windows. These days, VMWare Fusion and Parallels Desktop are two very mature virtualization products that allow clients to run virtually any Windows software on the Mac. Right now I have a client reporting an issue under VMWare Fusion 5. It’s an important client, so I’d better support him. My options to reproduce the issue are to buy a Mac Mini, or try out a virtual mac hosting service.

My Requirements

I need the following things

  • Root access. I need to install the software I want to test, including VMWare Fusion.
  • OS X Mavericks. I need the latest version of the operating system.

I’m reviewing these services based on my needs of occasional full speed access for testing. If you’re after a dedicated server for an application like web hosting, you will have different needs than I have.

The Roundup

Here’s a list of all of the services that I could find offering roughly the kind of service I’m looking for.

Some of these services offer cheaper plans with a more limited feature set (e.g. no root access). I’ve chosen in most cases to review the cheapest plan that offers Mavericks and that offers root access to your VM.

Service Notes
xcloud
xcloud.me
Plans from $85 USD/month (priced in swiss francs).
The rolls royce of virtual macs.
Does everything, the only downside is that TeamViewer is not preinstalled.
Mac Mini Cloud
macminicloud.net
Starts at USD $~150/month, which is somewhat sneakily described as a free tier. I didn’t see anything to recommend this over the other services I trialed, so I ruled this out based on the high cost and sneaky pricing.
Mac in Cloud
macincloud.com
1 day free trial is “included in each subscription”. Has a “pay as you go plan”, at $30 for 30 hours, which is great for an occasional user like myself. Does not include root access. Sounds good otherwise, but not suited to my needs so I didn’t try it. If you need specific software installed, you need to contact support to get it done. There’s a reasonable amount of positive media floating around about this mob, so I wouldn’t hesitate to try it.
vmOSX
virtualmacosx.com
Free trial by email application.
$40 USD/month for OSX Mavericks and root access. The plan is called Virtual Personal Desktop Advanced. They have cheaper plans for OSX Mountain Lion, and for non-root access. For your $40/month, you get 40gb storage, 1536 MB RAM and 2 virtual CPU cores. Virtual Machines are preconfigured with TeamViewer for remote access.I filled out the email form to request a trial, but at the time of publication they had not responded. There is some positive media floating around. They did however point out that they did not support running VMWare Fusion on their environment.
XCodeClub
xcodeclub.com
A one man show. For $35/month subscription, or $55 once off for a single month, you get 42gb storage (20gb free after OS install), 2gb RAM, and 2 virtual CPU cores. Daniel very politely offered a 1 hour trial. I was going to take him up on that, but I spent longer than that just getting connected to another service, so I wont bother just now. While I didn’t try it out, xcodeclub is highly spoken of and good value, and I wouldn’t hesitate to use this service. The only downside is the one-man-show aspect. VMs are preconfigured with TeamViewer.

I had every intention of trying out xcodeclub, vmosx and xcloud. However due to time constraints I ended up testing xcloud only.

xcloud

xcloud is an offering from innofield who also provide the “Flow App Engine” (flow.ch).

Three plans are offered, all include Mavericks and root access. The cheapest one (“mini”) is 79 CHF, which is USD $~85/month at today’s exchange rates. For that you get 80gb storage, 2gb RAM, and 2 x 2.3GHz CPU cores.
No trial offered, but I asked for a 1 week trial via their contact form anyway. They were happy to oblige. I’ll be trailing a “pro” instance with 160gb storage, 4gb RAM and 4 virtual CPU cores.

From the customer console, I am able to start and stop the virtual instance.

API Capabilities

A support engineer was kind enough to explain some of the capabilities of the snapshot API.

Create Snapshot: 
https://cp.innofield.com/api/v1/vm/snapcreate/{VMNAME}/{SNAPNAME}/token/{APIKEY}
Revert Snapshot: 
https://cp.innofield.com/api/v1/vm/snaprevert/{VMNAME}/{SNAPNAME}/token/{APIKEY}
Remove Snapshot: 
https://cp.innofield.com/api/v1/vm/snapremove/{VMNAME}/{SNAPNAME}/token/{APIKEY}

This is just as powerful as AWS or Azure, but it’s for Macs! Brilliant.

This API seems to be undocumented, I expect that this is a work in progress and that we’ll see more of this soon.

Documentation

Xcloud provides excellent documentation for some of the basics. The getting started guide covers every operating system you might consider connecting from. Some of the more common problems you might expect to encounter are well documented.

Connecting to the Instance

I imagine if you’re connecting from a Mac in the first place, this whole thing goes much smoother.

Right from the console, there’s a button to launch an HTML5 VNC client. Unfortunately, I couldn’t get the HTML5 client to work. I tried Chrome, IE and Firefox. I got this error in every case.

Failed to connect to server (code: 1006)

I was able to connect just fine with TightVNC, however. My experience was pretty slow, but I attribute this to my own ADSL connection rather than anything to do with xcloud. This was just painfully slow.

To resolve this, the Xcloud documentation suggests installing TeamViewer. So I started an AWS windows instance, RDPd to it, installed TightVNC there, VNCd to the xcloud instance (RDP-to-AWS-to-VNC-to-xcloud is actually reasonably fast), installed TeamViewer, then connected via TeamViewer. Phew! Happily, this all works just fine.

The only improvement to this would be if xcloud installed TeamViewer by default. VNC from Windows is just too slow.

Testing

Network connection seems more than acceptable. I was able to download a 4gb ISO from MSDN in less than 20 minutes.

I have full root access, so installing VMWare Fusion was not a problem. I couldn’t install 64-bit Windows, but the 32-bit version went in just fine.

The performance of this VM seems to be very good. It has no problem running a VM within a VM. My performance bottleneck is clearly the Internet connection speed to my own office.

Support

Xcloud support were quite responsive, and provided useful and timely answers to all of my questions.

Conclusion

Budget conscious? Xcodeclub might work out for you. But if you want a real killer go with xcloud for a few dollars more. You wont be disappointed.

 

Lookup and Verify your DNS Records

Lookup and Verify your DNS records from a Windows computer. How to check that you’ve set up your DNS records correctly using the nslookup tool installed with Microsoft Windows.

Verify your SRV records.

I followed this MSDN entry to put this example together.

In this example, I’m verifying that my DNS records have been configured correctly for using Microsoft Lync with Microsoft Office 365.

Execute these commands, substituting the record names for those you wish to verify.

nslookup
set type=srv
_sip._tls.contoso.com
_sipinternaltls._tcp.contoso.com

 

nslookup

 

Verify your TXT Records

Execute the following commands, substituting “contoso.com” for the domain you wish to verify.

nslookup
set type=txt
contoso.com

In this screenshot, I ran these commands on one of my own domains.

txt

Install specific version of NuGet package

In my base project, I’m using Json.NET 6.0.3. I need exactly the same version in my derived project. The primary reference for the nuget console can be found on the nuget site, where all of the commands I use here are covered in detail.

First launch the Package Manager Console.

menu

Ensure that the project of interest is selected in the “Default Project” menu at the top of the console window.

Execute the `Get-Package` command to find the name of the package were interested in.

PM> Get-Package -ListAvailable -Filter json.net
...
Newtonsoft.Json 6.0.3 Json.NET is a popular high-performance JSON framework for .NET
...

From the output of this command, I see that the project I’m interested in is properly called “Newtonsoft.Json”, so I’ll use this name in the next command.

Execute this command to update Json.NET to version 6.0.3.

PM> update-package Newtonsoft.Json -version 6.0.3
No updates available for 'Newtonsoft.Json' in project 'AAA'.
Updating 'Newtonsoft.Json' from version '5.0.6' to '6.0.3' in project 'BBB'.
Removing 'Newtonsoft.Json 5.0.6' from BBB.
Successfully removed 'Newtonsoft.Json 5.0.6' from BBB.
Adding 'Newtonsoft.Json 6.0.3' to BBB.
Successfully added 'Newtonsoft.Json 6.0.3' to BBB.
Uninstalling 'Newtonsoft.Json 5.0.6'.
Successfully uninstalled 'Newtonsoft.Json 5.0.6'.

Easy, we’re done!

Install Oracle Java on uBuntu Command Line

Go to the Java download page, accept the license agreement, and copy the download URL for the version you want. Then, on the target machine, use wget with the following command (credit).

wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/7u55-b13/jdk-7u55-linux-i586.tar.gz

Then extract and install with the following steps (credit)


tar -xvf jdk-7u55-linux-i586.tar.gz
sudo mkdir -p /usr/lib/jvm
sudo mv ./jdk1.7.0_55 /usr/lib/jvm/
sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0_55/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0_55/bin/javac" 1
sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0_55/bin/javaws" 1
sudo chmod a+x /usr/bin/java
sudo chmod a+x /usr/bin/javac
sudo chmod a+x /usr/bin/javaws
sudo chown -R root:root /usr/lib/jvm/jdk1.7.0_55

If we’re installing 32-bit Java on a 64-bit machine, we need to do one more thing (source):


sudo aptitude install libc6-i386

We can test our install by executing the following command


ubuntu@localhost:~$ java -version
java version "1.7.0_55"
Java(TM) SE Runtime Environment (build 1.7.0_55-b13)
Java HotSpot(TM) Client VM (build 24.55-b03, mixed mode)

The next thing we might want to do is to set the JAVA_HOME environment variable (source)

cp /etc/environment environment.temp
echo JAVA_HOME=/usr/lib/jvm/jdk1.7.0_55 >> environment.temp
sudo mv environment.temp /etc/environment

Backup and Restore Subversion Repositories

I’m moving my repositories to GIT, so I need to backup my old Subversion (SVN) repositories before I remove them from the server. Here’s how.

Backup

This command dumps a subversion to a text file (source). This command is identical on Linux or Windows.

svnadmin dump [[~/my/repos/path]] > backup.dump

Restore

Like the backup command, this restore command is also identical on Linux.

svnadmin load [[~/my/repos/path]] < backup.dump

Backup All Repositories At Once

It’s a simple matter to backup multiple repositories with one command..

On Windows, put this code in a ‘.cmd’ file, and execute it from your Repositories parent directory (source).

dir /A:D /B> dirs.tmp
FOR /F %%i IN (dirs.tmp) DO (
"C:\path\to\subversion\bin\svnadmin.exe" dump %%i > %%i.svn
)

The Bash/Linux/’.sh’ equivalent is as follows (source):

#!/bin/bash
for f in *; do
test -d "$f" && svnadmin dump "$f" >"$f.svn"
done

Migrate SVN to GIT with history

I’m migrating by Subversion repositories to GIT hosted on Atlassian BitBucket.

Convert Usernames

First make the  “authors” file. We’ll need this to translate author names from SVN usernames into the syntax that GIT likes. Run this command from the root of your subversion checkout (source).

svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u >> ~/authors.txt

Edit the file (nano ~/authors.txt)  to change this (source):

So this:

jwilkins = jwilkins <jwilkins>

becomes this:

jwilkins = John Albin Wilkins <johnalbin@example.com>

Install the Svn2Git Tool

Install svn2git (source)

sudo apt-get install git-core git-svn ruby rubygems
sudo gem install svn2git

Perform Local Conversion

Now perform the conversion. This will create a git repository in the current directory, so create an appropriate working directory and cd into it before executing this command.

svn2git http://svn.example.com/path/to/repo --authors ~/authors.txt --username [[my_subversion_user_name]]

Push Changes

Then push your changes to your remote repository

git remote add origin git@bitbucket.myaccount/myrepo.git ### connect to the remote repo
git push -u origin --all && git push -u origin --tags ### push everything

Wrap Up

Have a good hard look over the converted repository before removing the original subversion repository.

Before you do remove it, take a backup as follows, and store it somewhere safe.

svnadmin dump [[~/my/repos/path]] > backup.dump

 

Install and Update TeamCity on uBuntu

These are my brief notes on installing TeamCity. You will probably find that they are incomplete, so take caution. I do update them whenever I update TeamCity itself, but that won’t mean they are perfect.

See also (from official documentation): installing, taking a backup, upgrading.

Installation

Install Java

This step is necessary on both server and agent.

Before you do anything else, you need to install Java. Done? OK.

Install Server Components

You’ll want these packages:

sudo apt-get update
sudo apt-get install mysql-server
sudo apt-get install apache2 libapache2-mod-php5 phpmyadmin
sudo php5enmod mcrypt
sudo apache2ctl restart

 

Set Environment Variables

This step is only necessary on the server.

Here I’m setting an environment variable for all users. This is TEAMCITY_DATA_PATH, which is Teamcity’s working directory. (ref).

cp /etc/environment environment.temp
echo TEAMCITY_DATA_PATH=/var/teamcity/buildserver >> environment.temp
sudo mv environment.temp /etc/environment

Make a user

This step is necessary on both server and agent.

I like to run TeamCity under a dedicated user account.

sudo useradd -r -s /bin/false teamcity

Download the file 

Download and extract the latest build of TeamCity (from here).

wget http://download.jetbrains.com/teamcity/TeamCity-8.1a.tar.gz #### download build
/var/teamcity/TeamCity/bin/teamcity-server.sh stop #### stop the server if it's already running
tar -zxvf TeamCity-8.1a.tar.gz #### extract the files
sudo mkdir -p /var/teamcity
sudo rm -rf /var/teamcity/TeamCity ### remove any previous version
sudo mv TeamCity /var/teamcity/ #### deploy files, overwriting old files.
sudo chown -R teamcity:teamcity /var/teamcity #### set ownership of the data directory

Make an init script

Here we configure the TeamCity service to start and stop with the server. You only have to do this once

sudo cp /etc/init.d/skeleton /etc/init.d/teamcity
sudo nano /etc/init.d/teamcity

Set these properties:

DESC="TeamCity Server"
NAME=teamcity
DAEMON=/var/teamcity/TeamCity/bin/teamcity-server.sh

Near the top of the file, add this command (only on the server):

export TEAMCITY_DATA_PATH="/var/teamcity/buildserver"

Replace the contents of do_start() with:

start-stop-daemon --start -c teamcity --exec /var/teamcity/TeamCity/bin/teamcity-server.sh start
return 2

Replace the contents of do_stop() with:

start-stop-daemon --start -c teamcity --exec /var/teamcity/TeamCity/bin/teamcity-server.sh stop
return 0

Save the file

Configure the daemon to start automatically:


ubuntu@localhost:$ sudo chmod +x /etc/init.d/teamcity
ubuntu@localhost:$ sudo update-rc.d teamcity defaults
Adding system startup for /etc/init.d/teamcity ...
/etc/rc0.d/K20teamcity -> ../init.d/teamcity
/etc/rc1.d/K20teamcity -> ../init.d/teamcity
/etc/rc6.d/K20teamcity -> ../init.d/teamcity
/etc/rc2.d/S20teamcity -> ../init.d/teamcity
/etc/rc3.d/S20teamcity -> ../init.d/teamcity
/etc/rc4.d/S20teamcity -> ../init.d/teamcity
/etc/rc5.d/S20teamcity -> ../init.d/teamcity

Now the TeamCity server will start whenever the system boots up.

Initialize Teamcity

sudo service teamcity start

Launch a webbrowser to hostname:8111

Click Proceed.

Set up MySQL

stop teamcity:

sudo service teamcity stop

Install the Driver (download page):

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.30.tar.gz
tar -zxvf mysql-connector-java-5.1.30.tar.gz
mkdir -p /var/teamcity/buildserver/lib/jdbc/
sudo cp mysql-connector-java-5.1.30/mysql-connector-java-5.1.30-bin.jar /var/teamcity/buildserver/lib/jdbc/
rm -rf mysql-connector-java-5.1.30 mysql-connector-java-5.1.30.tar.gz
sudo chown -R teamcity:teamcity /var/teamcity/buildserver/lib/jdbc/

Make a database “teamcity” and user “teamcity” with full privileges on the database (e.g. using phpmyadmin).

Save the file

Start teamcity:

sudo service teamcity start

You’ll need an authentication token, which you’ll find in /var/teamcity/TeamCity/logs/teamcity-server.log

Update TeamCity

Download update

Get the latest version of the MySQL connector (from here).


wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.29.tar.gz #### download latest mysql connector
tar -zxvf mysql-connector-java-5.1.29.tar.gz
sudo cp mysql-connector-java-5.1.29/mysql-connector-java-5.1.29-bin.jar /var/teamcity/buildserver/lib/jdbc/
rm -R mysql-connector-java-5.1.29/ #### remove temporary mysql files
sudo chown -R teamcity /var/teamcity/buildserver/lib/jdbc/ #### set ownership

Download the latest build of TeamCity (from here).

wget http://download.jetbrains.com/teamcity/TeamCity-8.1a.tar.gz #### download build
sudo service teamcity stop #### stop the server if it's already running
tar -zxvf TeamCity-8.1a.tar.gz #### extract the files
sudo rm -rf /var/teamcity/TeamCity ### remove any previous version
sudo mv TeamCity /var/teamcity/ #### deploy new version
sudo chown -R teamcity:teamcity /var/teamcity #### set ownership of the data directory

Start the TeamCity Server

ubuntu@brick:~$ sudo service teamcity start
[sudo] password for ubuntu: ********
Using CATALINA_BASE: /var/teamcity/TeamCity
Using CATALINA_HOME: /var/teamcity/TeamCity
Using CATALINA_TMPDIR: /var/teamcity/TeamCity/temp
Using JRE_HOME: /usr/lib/jvm/default-java
Using CLASSPATH: /var/teamcity/TeamCity/bin/bootstrap.jar:/var/teamcity/TeamCity/bin/tomcat-juli.jar

Tail the logs while you’re waiting for the server to start up.

tail -f /var/teamcity/TeamCity/logs/teamcity-server.log

Browse to your TeamCity web interface. You will be told to wait.

 

Installing a Build Agent

Here are the steps to install a Ubuntu build agent (on a separate machine than the server)

Download the binaries from your server.

wget http://your-teamcity-server:8111/update/buildAgent.zip
7z x buildAgent.zip -o"/var/teamcity/buildAgent"

Configure the “teamcity” user

sudo useradd -r -s /bin/false teamcity

Extract the binaries and set directory permissions.

sudo mkdir -p /var/teamcity
sudo 7z x buildAgent.zip -o"/var/teamcity"
sudo chown -R teamcity:teamcity /var/teamcity #### set ownership of the data directory

Set up a daemon, so that the build agent starts and stops with the instance.

sudo cp /etc/init.d/skeleton /etc/init.d/teamcity
sudo nano /etc/init.d/teamcity

Set these properties:

DESC="TeamCity Build Agent"
NAME=teamcity
DAEMON=/var/teamcity/bin/agent.sh

(In this step, I also add “/usr/local/bin” to PATH for my own setup)

Replace the contents of d_start() with

start-stop-daemon --start -c teamcity --exec $DAEMON start
return 2

Replace the contents of d_stop() with

start-stop-daemon --start -c teamcity --exec $DAEMON stop kill
return 0

Specify the server url in /var/teamcity/conf/buildAgent.properties

cp /var/teamcity/conf/buildAgent.dist.properties /var/teamcity/conf/buildAgent.properties
nano /var/teamcity/conf/buildAgent.properties

Now start the service with

service teamcity start

Expand disk on ESXi Virtual Machine running Ubuntu – the easy way!

My TeamCity server is running on a virtual machine with a ubuntu server OS, under ESXi. It’s currently pretty low on free disk space. I’d like to improve that situation. I’ll do that by expanding the disk.

Before you begin take a snapshot in case you mess something up. I’ll assume you already know how to do that, so I’ll proceed to actually expanding the disk. The first bit is easy, and is done in the vSphere Client.

I spent hours trying to do this on the command-line. Eventually I relented, and did it the easy way! So I’ll walk you through how you can also do it the easy way.

Download gparted.
Upload it to your esxi storage.

  • Shutdown your VM (either via SSH or via the vSphere Client)
  • Launch the vSphere Client and connect to the host.
  • Right-click on the VM’s name.
  • Select Edit Settings.
  • Select the hard disk you want to expand.
  • Increase the Provisioned Size to the desired capacity.
  • Click OK.

Boot to gparted

gparted

Look carefully, you’ll see that /dev/sda5 is located inside of /dev/sda2. I’ll need to expand sda2 before I can expand sda5.

* Select

gparted2

Click Apply

I feel like I achieved something there.

Now back to the command-line

I ran this command to resize the physical volume.

ubuntu@brick:~$ sudo pvresize /dev/sda5
Physical volume "/dev/sda5" changed
1 physical volume(s) resized / 0 physical volume(s) not resized

Then I ran this command to resize the logical root volume

ubuntu@brick:~$ sudo lvresize -l +100%FREE /dev/mapper/brick-root
Extending logical volume root to 118.76 GiB
Logical volume root successfully resized

Then I ran this command to resize the file system on the root partition. This took about 6 seconds.

ubuntu@brick:~$ sudo resize2fs /dev/mapper/brick-root
resize2fs 1.42.5 (29-Jul-2012)
Filesystem at /dev/mapper/brick-root is mounted on /; on-line resizing required
old_desc_blocks = 5, new_desc_blocks = 8
The filesystem on /dev/mapper/brick-root is now 31132672 blocks long.

At this point it seems like we’re OK, my root device is now 120gb.

References:

http://serverfault.com/questions/28989/fdisk-l-like-list-of-partitions-and-their-types-for-lvm-logical-volumes

http://ryandoyle.net/posts/expanding-a-lvm-partition-to-fill-remaining-drive-space/