Mounting shared folder - Virtualbox

Download the virtual box from https://www.virtualbox.org/wiki/Linux_Downloads.

Select import option from file menu to extract the vmdk from .ova file.

Setting up network -> preferences and and enable host-to-host

in the guest, run $ sudo dhclient to assign the ip address.

Mounting shared filesystem:

Settings -> shared directory-> add new shared directory “vmshared”

Devices -> insert guest addition CD image

mount -t vboxsf -o uid=1000,gid=1000 vmshared <mount-point>

Qemu tap network script

In the previous posts on Qemu networking, we have seen a manual procedure to setup up network access using Qemu tap device. Its a good way to get started but ideally we would like all these to be automated to ease our life. Here is a script that will automatically bring up the tap device and logically assign an IP address to that device. Name the script as qemu-ifup and place it under /etc directory. Qemu automatically picks up this script. Here is the qemu command line

$ qemu-system-x86_64 -hda cirros-0.3.0-x86_64-disk.img -netdev tap,id=net0,downscript=no -device e1000,netdev=net0

Script

$ cat /etc/qemu-ifup

#!/bin/bash                                                           
                                                                      
tap=$1
num=`echo $tap | sed 's/[a-zA-Z]*\([0-9]*\)/\1/'`                     

device_num=$(( $num + 1 ))

echo "tap = $tap, device_num = $device_num, num = $num"

#now generate a IP address
ip_addr=10.1.70.$device_num

ip link set $tap up


ip addr add $ip_addr/24 broadcast + dev $tap

echo 1 > /proc/sys/net/ipv4/ip_forward

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

iptables -I FORWARD 1 -i $tap -j ACCEPT

iptables -I FORWARD 1 -o $tap -m state --state RELATED,ESTABLISHED -j ACCEPT

[links]
* http://tech-jnaan.blogspot.in/2013/08/configuring-qemu-tap-interface-for.html
* http://tech-jnaan.blogspot.in/2013/08/setting-up-qemu-with-nat.html

Attaching an additional hard disk to Qemu

I am using Cirros as my primary hard disk to boot the operating system through Qemu. It has been very handy and useful until I faced with the space shortage problem. So, I had to look for a way to increase the disk space to continue with my work. That's when I came across the Qemu option hdb which provides a means to add a new hard disk.

To start with create a disk image of your choice. I will use Qcow2 image format to create the disk.

$ qemu-img create -f qcow2 drive.qcow 4G

Once we have created the disk image, we should pass this as part of the Qemu command line.

$ qemu-system-x86_64 -hda cirros.img -hdb drive.qcow

Inside guest VM

When we login into the vm, before we can start using our new disk, we have to partition the disk and format the partitions with a file system. Before we format the disk, we must find the new block device that the kernel created as part of the boot process.
$ ls /dev/sd*

By default, the name of the block device will be sdb. It is a raw disk, we must create partitions underneath it. You can use the system tool that you are comfortable with. I will use cfdisk.

$ cfdisk /dev/sdb

Create partitions according to your requirement. Then, write the changes to the disk and exit.

Next, we just format the new partition with a file system. I will use ext4 file system format to format the new partition.

$ mkfs.ext4 /dev/sdb1

We must mount the new partition to access the disk space in it.

$ mkdir /drive
$ mount -t ext4 /dev/sdb1 /drive

$ df

Now, the new mount point will appear as part of the output of the df command.

[links]
* http://wiki.qemu.org/Main_Page
* http://en.wikipedia.org/wiki/Qcow
* https://launchpad.net/cirros/+download

Gerrit

To those of you who are new to Gerrit, Gerrit[1] is a free, web-based team code review tool which can be used by members of a team to review each other's modifications in their source code using a web browser and approve or reject those changes. It integrates closely with Git.

Prerequisite

• Java is needed to install and run Gerrit. If you don't have Java installed, then download and install the latest JDK on your machine.

• Create a user named gerrit2 to host gerrit services.
  

  $ sudo adduser gerrit2

Installation

• Download the latest version of gerrit(http://code.google.com/p/gerrit/downloads/list). This document focusses on gerrit.2.2.2.

• To install gerrit, first change to gerrit user
  

  $ sudo su - gerrit

• Iinitialize with the batch switch enabled, so that we don't have to answer any questions.
  

  $ java -jar gerrit.war init --batch -d <site-path>

  site-path: lets assume that it is ~/gerrit_testsite

         This is automatically start gerrit on localhost running on port 8080.

Configuration

• You may wish to change the port on which gerrit runs on. To do this, edit the gerrit_testsite/etc/gerrit.config and replace the port number 8080 with the port of your choice. In this document we will stick with the port number 8080.

• Now restart the gerrit process.
  

  $ ~/gerrit_testsite/bin/gerrit.sh restart

Registering User

• Open the browser and enter the url www.localhost:8080 and it will prompt you to register/sign in. You can use open-id to sign in. In Gerrit, the first user to register becomes the default administrator.

• On the registering site, add the public key of the machine from which you 'll send patches to.

• Now you are all set. Try loging in using ssh, gerrit uses port 29418 for ssh access.
  

  $ ssh -p 29418 john@localhost

The authenticity of host '[localhost]:29418 ([127.0.0.1]:29418)' can't be established.

RSA key fingerprint is 35:72:c5:2d:0c:16:8a:68:04:ab:2f:51:f1:d0:0c:6b.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '[localhost]:29418' (RSA) to the list of known hosts.

***    Welcome to Gerrit Code Review    ****

Hi john, you have successfully connected over SSH.

Unfortunately, interactive shells are disabled.

To clone a hosted Git repository, use:

git clone ssh://john@localhost:29418/REPOSITORY_NAME.git

Connection to localhost closed.

Creating a Project

Gerrit gives the flexibility to either create a new project from scratch or upload an existing project to gerrit.

• To create a new project from scratch, create a new repository on the gerrit machine by passing the option empty-project.
  

  $ ssh -p 29418 john@localhost gerrit create-project --empty-project -name test-proj

  In our case, we created an empty project named test-proj.

• Alternatively, you can tell gerrit that you will be uploading an existing project. To do this, first create a project on the gerrit machine.
  

  $ ssh -p 29418 john@localhost gerrit create-project name test-proj

  This command will create a git repository on the gerrit machine. Behind the scenes, the above command runs 
  

  $ mkdir test-proj.git

  $ cd test-proj.git

  $ git init --bare --shared

  Once the project is created, add a remote repository reference to your local repository.
  

  $ git remote add origin ssh://john@localhost:29418/test-proj.git

  Now push your changes to the upstream repository
  

  $ git push origin

  Counting objects: 5, done.

  Writing objects: 100% (3/3), 248 bytes, done.

  Total 3 (delta 0), reused 0 (delta 0)

  To ssh://john@localhost:29418/test-proj.git

  ! [remote rejected] master -> master (prohibited by Gerrit)

  error: failed to push some refs to 'ssh://john@localhost:29418/test-proj.git'

The above error is because the user is trying to push the changes to the upstream repository directly, bypassing the gerrit's review process.

• Gerrit maintains a seperate change branch to which the new changes would be committed and made available for review to other users.
  

  $ git push origin HEAD:refs/for/master

Hear master is the branch name to which the changes have to be applied to. Replace it with the branch name to which you want to apply your changes to.

Clone Project

• Users who wish to clone the project must first register to the project and then clone using ssh.
  

  $ git clone ssh://smith@localhost:29418/test-proj.git

Permissions

• Now, go back to the gerrit admin page and click on the project link(http://localhost:8081/#/admin/projects/) to set permissions as to how the project will be maintained. Click on the project name which in our case is test-proj (http://localhost:8081/#/admin/projects/test-proj,access), click on the references link and add permissions from the 'add-permission' dropdown menu. For most cases, the following permissions should work

• Owner

• Push Merge Commit

• Label Code-Review

• Label Verified

• Submit

There are various types of groups like Administrator, Registered users, Anonymous, etc. Apart from this, you can explicitly a new group and add members to this newly created group and grant permissions to them.

• Now that the permssions are all set, users can clone the repo, make changes locally and push the changes to review.
  

  $ git clone ssh://steve@localhost:29418/test-proj.git

  $ cd test-proj

    ---- make changes and add the changed files to git ----

  $ git commit

  $ git push origin HEAD:refs/for/<branch-name>

  
  

[1] http://en.wikipedia.org/wiki/Gerrit_(software)

http://gerrit.googlecode.com/svn/documentation/

Setting up QEMU with a NAT

This is a continuation of my previous post which deals with setting up network connection for Qemu between host and guest. In this post we will learn to configure Qemu to access outside world.

There are a number of ways to configure Qemu to access internet like bridging, proxy arp, etc. In this post I will use tap device that we configured in the previous post and then make some changes to host iptables to forward the packets over the internet.

Nat

$ echo 1 > /proc/sys/net/ipv4/ip_forward

# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

# iptables -I FORWARD 1 -i tap0 -j ACCEPT

# iptables -I FORWARD 1 -o tap0 -m state --state RELATED,ESTABLISHED -j ACCEPT

Thats it. Verify now by pinging any of the outside connection in the guest. The simplest is,

$ ping google.com

Links
* http://tech-jnaan.blogspot.in/2013/08/configuring-qemu-tap-interface-for.html
* http://felipec.wordpress.com/2009/12/27/setting-up-qemu-with-a-nat/

Configuring Qemu tap interface for guest<-->host communication

I recently started playing with Qemu. One of the first things that I wanted to setup was a communication link between the host(physical machine) and guest(the vm). Qemu provides a number of techniques to achieve this. In this post I will explain about setting up network with tun/tap device.

It is easy to set it up when you know what it is. I spent couple of hours while I was fiddling with it. Read a lot of documents but dint find one which explained this completely and hence this post. I received a lot of help on the Qemu irc channel(#qemu on OFTC).

I assume that Qemu is already installed on your machine and you also have installed OS on the virtual disc. If not you can download the latest source code and install Qemu from it. Also, you can download a sample virtual disk. Once all these are ready, follow the steps below.

First, start the qemu emulator on the host machine. 

$ qemu-system-x86_64 -hda cirros-0.3.0-x86_64-disk.img -netdev tap,id=net0,script=no,downscript=no -device e1000,netdev=net0

The above command will start the Qemu with KVM. The most interesting parts of the command is the -netdev option. Here, we instruct the Qemu to use tap interface to provide networking capability. Once we have started the Qemu, lets configure the host to send/receive packets. On the host, check the name of the tap device that Qemu is using to communicate the packets. It picks up the first unused device. 

 $ ip addr show

Once we know the name of the tap device, we should bring the interface up and also assign an IP address to it. Here, we are assuming the interface name as tap0. 

$ ip link set tap0 up

Now assign IP address to the tap interface 

$ ip addr add 10.1.70.1/24 broadcast + dev tap0

Once we do this, we are all set to send/receive the packets from the host. Now, we must configure the guest to send/receive the packets over the tap device. Once the guest machine is up, assign an IP address to the guest machine from the same subnet. 

$ ip addr add 10.1.70.2/24 broadcast + dev eth0

Since we had already instructed Qemu to use the tap device, it will send the packets through it. Now, check for connectivity by running

$ ping 10.1.70.1

The above IP address is the IP that we assigned to the tap device. The interesting thing to note here is that when we try to access the IP address of your host machine, ping command will return with network unreachable error message. The reason being that the guest machine doesn't know the route to follow to reach the host when this IP address is given. Let us add the necessary rule which would be used to send/received the packets.

$ route add default gw 10.1.70.1 dev eth0

Now, we can use the IP address of tap or host to access the host machine.

Links:

* http://wiki.qemu.org/Main_Page

* http://en.wikipedia.org/wiki/TUN/TAP

* https://launchpad.net/cirros/+download
* https://people.gnome.org/~markmc/qemu-networking.html

Mutexes VS Semaphore

Mutex: Mutex is a program object that is typically used to serialise access to a section of re-entrant code that cannot be executed concurrently by more than one thread. A mutex object only allows one thread into a controlled section, forcing other threads which attempt to gain access to that section to wait until the first thread has exited from that section."

Semaphore: A semaphore is like an integer, with three differences:

• When you create the semaphore, you can initialize its value to any integer, but after that the only  operations you are allowed to perform are increment (increase by one) and decrement (decrease by one). You cannot read the current value of the semaphore.
• When a thread decrements the semaphore, if the result is negative, the thread blocks itself and cannot continue until another thread increments the semaphore.
• When a thread increments the semaphore, if there are other threads waiting, one of the waiting threads gets unblocked.

To say that a thread blocks itself is to say that it notifies the scheduler that it cannot proceed. The scheduler will prevent the thread from running until an event occurs that causes the thread to become unblocked. In the tradition of mixed metaphors in computer science, unblocking is often called  as€œWaking.

From the definitions, it is very much clear that mutex contains binary value either True or False but a semaphore contains an integer value which holds those many number of keys to get that resource.

To further understand these two, lets consider an example

• Mutex is a key to a toilet. One person can have the key - occupy the toilet-at the time. When finished, the person gives (frees) the key to the next person in the queue.
• Semaphore on the other hand is analogous to the he number of free identical toilet keys. Say we have four toilets with identical locks and keys. The semaphore count - the count of keys - is set to 4 at beginning (all four toilets are free), then the count value is decremented as people are coming in. If all toilets are full, ie. there are no free keys left, the semaphore count is 0. Now, when eq. one person leaves the toilet, semaphore is increased to 1 (one free key), and given to the next person in the queue.

One of the often asked question is "what is the difference between a mutex and a binary semaphore?"
  By binary semaphore I mean that the value store in the semaphore is 1. So its similar to the mutex. But there are quiet a few good characteristics that distinguish the two.

• Mutexes can be recursive (so reentrant lock works), and Semaphores are not.
• One significant difference - a semaphore may be procured and vacated by any thread in any sequence (so long as the count is never negative), but a mutex may only be unlocked by the thread that locked it. Attempting to unlock a mutex which was locked by another thread is undefined behavior
• The Mutex class enforces thread identity by virtue of owners, so a mutex can be released only by the thread that acquired it. In contrast, the Semaphore class doesnot enforce thread identity.

Now lets understand what are recursive mutexes:
POSIX allows mutexes to be recursive. That means the same thread can lock the same mutex twice and won't deadlock. Of course it also needs to unlock it twice, otherwise no other thread can obtain the mutex. Not all systems supporting pthreads also support recursive mutexes, but if they want to be POSIX conform, they have to.

Now, you may ask why on the earth would I call lock on the same mutex. To find the answer please refer links 1 and 2. It has pretty good explanation and I dont want to duplicate the effort.

There is something called as Futexes aswell, these are Fast user level mutexes, pthread library on linux implements Futexes, so all your user level programs that spawn thread are already using Futexes. Futexes have an edge over mutexes and hence are very useful.

[] http://stackoverflow.com/questions/2415082/when-to-use-recursive-mutex
[] http://stackoverflow.com/questions/187761/recursive-lock-mutex-vs-non-recursive-lock-mutex
[] http://blog.feabhas.com/2009/09/mutex-vs-semaphores-%E2%80%93-part-2-the-mutex/
[] http://web.itu.edu.tr/kesgin/mul06/intel/instr/cmpxchg.html