Formatting a filesystem using loopback Devices

When it comes to selecting a filesystem, Linux provides its users with a vast set of filesystem to choose from. There are many filesystems which are part of the linux kernel and there are others that are developed over fuse layer and the list is constantly growing everyday. When ever a new filesystem is released you might not want to format it on a partition (because of limited disk space, or to avoid disk fragmentation etc). One of the best method to try out a new filesystem is to format it on a loop back device and mount it on a directory. In this post I will demonstrate how to create a loopback device and then format it with ext4 (you can use any Filesystem of your choice) and the mount it on a directory.

The very first step is to create a file which will act as a psuedo partition so choose the size appropriately. There are two ways of doing it

  * using dd command

  * using truncate command

I prefer truncate over dd  because truncate creates a sparse file so the file does not consume any

disk space untill you add data into it but file created using dd will consume the disk space. To confirm that, 

truncate --size=1GB file-1GB

  Now check the output of

ls -l file-1GB
du -h file-1GB

ls will report the file size as 1GB but du will report the file size as 0. So, it can be confirmed that the file doesn't use any disk space.

  Now try

dd if=/dev/zero of=file-1GB bs=1KB count=1000000

  Now check the output of ls and du.

So, now you have created partition but the system doesn't recongnize it. So lets attach it to a device file.

losetup -f

Let's assume that it retured /dev/loop0, now run

losetup /dev/loop0 file-1GB

Okay. Now all set, choose a filesystem of your choice and format the device. I choose ext4 to demonstrate it.

mkfs.ext4 /dev/loop0

Now mount the device on a directory.

mkdir /my-fs

mount -t ext4 -oloop /dev/loop0 /my-fs.

Thats it. You can confirm it by running

df

your Filesystem will be listed there. Happy hacking :)

Note: When you reboot the system your filesystem will not be displayed by df command. You can either add the entry into /etc/fstab or write a script that will attach the file-1GB to a device and run the mount command. A sample script can be found here. 

Explore VIM

"Give me six hours to chop down a tree and I will spend the first four sharpening the axe.”
– Abraham Lincoln

I start the post with an interesting note. We must have our tools ready before we start our work, it makes our work more fun otherwise it becomes a pain. A similar case is with the programmers. They spend most of there time in front of the terminal either writing new code or refactoring the old code. Having an environment customized as per one's need can make a huge difference.

Vim is one of the pieces of software that I use almost everyday. I have been a VIM user since the college days. I started with vi, When I started using it I was a bit annoyed because I had to remember all those commands to browse through the file, changing the modes to move from one line to another was a pain. But when I first used VIM, I was like I have been waiting for something like this because it had all the features that were provided by vi and it had many more. It was definitely more powerful than vi. In this post I will share some of the features which I personally have found to be very useful.

• In VIM searching a word in the given file is achieved  by typing "/" in the command mode. But adding the following lines to your .vimrc file can make a huge difference.
  
  To move the cursor to the matched string while typing the search pattern, set
  

  set incsearch

   With 'ignorecase', searching is not case sensitive:
  

  set ignorecase

   If 'ignorecase' is on, you may also want:
  

  set smartcase

  When 'ignorecase' and 'smartcase' are both on, if a pattern contains an uppercase letter, it is case sensitive, otherwise, it is not. For example, "/The" would find only "The", while "/the" would find "the" or "The" etc.
  
  We can also search for a word in the file by placing the cursor on the word to be searched and pressing "*", this will take you to the next instance of the word in the file.
  Press "n" to move to the next instance of the searched word and "N" to goto the previous instance of the searched string.

• Searching and replacing is one of the very common task. One can achieve this in VIM by typing
  

  :%s/search-string/replace-string/

  This will search for "search-string" and replace only the first matched instance of that string with "replace-string. To replace all the instances of the "search-string" add "g" at the end of the command.
  

  :%s/search-string/replace-string/g

  To search and replace only in few lines, provide the range of lines in the command. To do a find and replace only in lines 50-100, run
  

  :50,100s/search-string/replace-string/g

  To delete "search-string", donot pass any parameter in the replace-string, it will delete the "search-string". To delete all the "search-string" in lines 50-100, run
  

  :50,100s/search-string//g

• Type "gf" in command mode to open the file containing the word the cursor currently points to.

• Most of the time while browsing the code we feel the need to have more than one file opened in the terminal, either we open each file in a seperate terminal or close the current file and open the other. But VIM has tabs. Yes, it provides tabs just the way the browser's provide tabs. To open a file in the new VIM tab, type
  

  :tabedit file_name

  
  
  Use "ctrl+alt+pgup/pgdn" to traverse between the VIM tabs.
  
  Type :help tabedit in the command mode to explore more.

• You use some words frequently in your document and wish there was a way that it could be quickly filled in the next time you use the  same word, VIM provides autofill feature. Press
  

  ctrl+n

  to achieve this.
  
  
  

• To add line numbers to the file type
  

  :set number

  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  

• Good programming is all about making the code more readable, one of the ways  of achieving it is through indenting the code. Most of the time coping a piece of code from one block to another breaks this, VIM provides a way to perform some actions on the selected block. Press shift+v to select a block, as shown in the figure below:

     

Selecting a block with shift+v

       
          Once a block is selected we can
              *  Move the block 8 spaces to the left by pressing shift + < or 8 spaces to the right by pressing
                  shift + > so as to indent the code
              *  Copy the block by pressing "y" and then pasting it somewhere else by pressing "p".
              *  Search and replace a word only in the selected block, press
                  ":'<,'>s/search_string/replace_string/g"


        Or we can select column's by pressing "ctrl + v" as shown in the figure below.

         

Selecting a Column by pressing cntl + v

• This is my favorite, VIM allows to split the page horizontally and vertically. This is of great use when we want to view two different files at a time. To split the file vertically press :vsp in the command mode.
  
  

  

  Vertical split

          Press ":sp" to split the screen horizontally. Press ":help vsp" to explore more about how to
          switch between the window etc.

• Upper case and Lower case: VIM provides an easy way to change the case of the line to UPPER case or LOWER case. Here is the command,
  
            gUU or guu 
  
  Place the cursor on the line which you want to convert the case and in the command mode type these letters. gUU will convert the text to upper case and guu will convert the text to lower case.
  
  You can also flip the case of a set of character with the help of "~" command. To explore more on this type
  
          :help gUU

• To open files located in other directories, you can use the full or relative paths such as 

:e file name 

• redo and undo: In Vim to undo the last command press
  
            undo- u
  
  and to redo the last command press
  
            ctrl+r
  
  in command mode
  
• When you copy some code from the browser or other editors like gedit and paste it into Vim, you may loose the alignment. To keep the alignment intact, type
  
            :set paste
  
   and then paste the code. To change back to normal mode, type
  
            :set nopaste
  
  in command mode.
  
• To find more help on Vim type ":help" command mode.
  
• Find below a few other options that can be set in your .vimrc

            * set scrolloff=100
            * set cursorline
            * set autoindent " always set autoindenting on
            * set backup " keep a backup file
            * set backupdir=~/bkp/vim
            * set history=50 " keep 50 lines of command line history
            * set background=dark

Some useful links:
[1]  http://vim.wikia.com/wiki/Vim_Tips_Wiki
[2]  https://github.com/mja054/Random-tweeks/blob/master/vimrc - to find a sample vimrc
[3] http://dan.hersam.com/docs/vim.html
[4] http://vimdoc.sourceforge.net/htmldoc/options.html#'shiftwidth'

Segmentation Fault

A segmentation fault or bus error occurs when the hardware notifies a operating system about a memory access violation. On receiving the notification the OS sends a signal to the process which caused the exception. Now its upto the process receiving the exception to decide how it would like to handle this exception. By default the process receiving the signal dumps its state in to a core file and terminates but the default signal handler can be overridden to customize how the signal is handled.

There are many ways to get a segmentation fault, at least in the lower-level languages such as C(++). A common way to get a segmentation fault is to dereference a null pointer:

int *p = NULL;
*p = 1;

Segmentation fault also happens when you try to write to a portion of memory that was marked as read-only:

char *str = "Foo"; // Compiler marks the constant string as read-only
*str = 'b';              // Which means this is illegal and results in a segfault

Accessing Dangling pointer also causes segmentation faults like here:

char *p = NULL;
{
    char c;
    p = &c;
}
// Now p is dangling

The pointer p dangles because it points to character variable c that ceased to exist after the block ended. And when you try to dereference dangling pointer (like *p='A'), you would probably get a segmentation fault.

Another reason for segmentation fault is to recurse without a base case, which causes a stack overflow:

int main() { main(); }

Segmentation fault also happens when accessing a region of memory already freed

Whenever segmentation faults happens, a core is generated by the process in the current directory. By default, most of the Linux distros disable the creation of core. So if you dont find the core file run the following command to enable the creation of core files.

$ ulimit -c unlimited

 Generally core files are created as core.<pid>, pid is the pid of the process that created the core.

Running the above command will set ulimit to unlimited on that terminal. So when you start a new terminal it would not be set. So to make it default in all the new terminals add it into ~/.bashrc.

But to make sense out of the core dump you must compile the program with debugging symbols. Lets take an example program to understand how to use the core dump to the fullest.

$ cat hello.c

#include <stdio.h>
int
main ()
{
        int *p = NULL;
        *p = 2;
        return 0;
}

 Now enable debugging symbols while compiling

$ gcc -g -O0 hello.c -o hello

Now when you run this program, you will receive the following error

$ ./hello
Segmentation fault (core dumped)

You will find that a core file is created in the current directory.

Now run the core with the debugger (on most of the distros gdb is provided if not install it)

$ gdb hello core.5123

It will display the line in the program that caused the error. There are many other options that you can use with gdb to debug your issue, we will visit that in future posts.

Some of the best practices to follow while programming in C/C++ is to 

• Always check for NULL before dereferencing the pointers. Example
  
           if (ptr == NULL)
                     //return with error
  
• Also check for NULL after the call to dynamic memory allocation functions like malloc, calloc, new etc. Example
  
          ptr = (int *) malloc (10 * sizeof (int));
          if (ptr == NULL)
                     //return with error

References

* http://collectd.org/wiki/index.php/Core_file

* http://stackoverflow.com/questions/2346806/what-is-segmentation-fault

* http://en.wikipedia.org/wiki/Segmentation_fault

SSH: "Remote host Identification has changed" error

SSH is the most common and widely used method for remote login. It is better the Telnet because it provides authentication mechanism to secure your machine (If you dont want to enter your password each time when you login, checkout the previous post on passwordless-ssh-login-howto). But, sometimes one may find that the machine that we used to login sometime back is refusing to authenticate. If the error that you found is

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
9b:8e:77:08:7c:8f:2d:f7:0b:4e:dc:d3:98:29:dd:ec.
Please contact your system administrator.
Add correct host key in /root/.ssh/known_hosts to get rid of this message.
Offending key in /root/.ssh/known_hosts:8
RSA host key for 192.168.1.8 has changed and you have requested strict checking.
Host key verification failed.

Just delete the file ~/.ssh/known-hosts.

> rm ~/.ssh/known-hosts

 Thats it, now try running ssh at the terminal it will succeed.

Ubuntu: Creating a bootable Pendrive

 In this post, I will demonstrate how to create a bootable pendrive from an .iso image on ubuntu. Creating a bootable pendrive is very easy in ubuntu. If you are using ubuntu 10.10, then goto

System->Administrator->Startup Disk Creator

Select Startup Disk Creator it will prompt for password if you are not root, enter the password and it starts. It looks like this

Startup Disk Creator

Once you have started the Startup Disk Creator follow these steps to create a bootable Pendrive:

1. click on  "other" button and select the location where you have stored the .iso file.


2. Then if you want to erase all the data on the pendrive then press on "Erase Disk" button and wait for it complete. Skip this step if already empty.


3. Select "Make Startup Disk" button.







Thats it. Once it completes you have a bootable pendrive which you may use it for

• liveCD
• installing a fresh distro installation

I have tested this for creating a bootable pendrive from ubuntu-11.04.iso on ubuntu-10.10. If you happen to use it for other distros then do post your results

mencoder: Converting .ogv to .avi

recordMyDesktop produces videos in .ogv format. Youtube and many other websites fail to play the .ogv video. Use mencoder to convert the .ogv video. mencoder is a simple movie encoder, designed to encode MPlayer-playable movies to other  MPlayer-playable formats.

In ubuntu, install the mencoder using

>sudo apt-get install mencoder

once installed, use the following command to covert the .ogv file to .avi file.

>mencoder video.ogv -ovc xvid -oac mp3lame -xvidencopts pass=1 -o video.avi

 There are many more options to mencoder, read the man page for more info.

gtk-recordMyDesktop

The Chinese proverb "A picture is worth thousand words" fits perfectly here, so I will be demonstrating by showing a video.

The front end makes use of the commandline options to perform the task, which means that the things we achieve through the gui can also be done through the commandline.

Prev