Saturday, February 21, 2009

Local snapshots

Recently I've started using rsnapshot, and have found it pretty useful. Rsnaphsot uses librsync to create snapshots of various directories.

On my systems I typically configure backups of:

backup /etc/ localhost/
backup /usr/local/ localhost/
backup /var/lib/rpm localhost/

Drop in cron entries into /etc/crontab:

# Backup filesystems 0 */4 * * * root /usr/bin/rsnapshot hourly 50 23 * * * root /usr/bin/rsnapshot daily 40 23 1,8,15,22 * * root /usr/bin/rsnapshot weekly 30 23 1 * * root /usr/bin/rsnapshot monthly

And get snapshots under /.snapshots

daily.0 daily.1 daily.2 daily.3 daily.4 daily.5 daily.6 hourly.0 hourly.1 hourly.2 hourly.3 hourly.4 hourly.5 monthly.0 weekly.0 weekly.1 weekly.2 weekly.3

It doesn't take up much space. If I wanted to I could store the snapshot remotely. And it's been helpful when I needed to remind myself what I did on some host that may have caused issues.

Saturday, January 10, 2009

Certificate Based Authentication In Perl

There have been a few times where I've needed to create a remote service that would run from cron but I couldn't use Kerberos and didn't want to have an exposed password. Certificated based authentication is a handy thing to have in your toolbox when the need arises. The examples below are just slight modifications to the example client and server that come with the excellent IO::Socket::SSL module. Hopefully the example and comments make the code speak for themselves. create_certs.sh
     1  #!/bin/bash
     2  # ----------------------------------------------------------------------------
     3  # create_certs.sh - Create a self signed certificate and create a client
     4  #                   cert that is signed by that cert.
     5  #
     6  # See Also: http://www.madboa.com/geek/openssl/#cert-self
     7
     8  # ----------------------------------------------------------------------------
     9  # GLOBALS
    10
    11  CERTDIR=certs              # Directory to store our certs
    12  C='US'                     # Country
    13  S='New Jersey'             # State
    14  L='lost'                   # Location (here I'm using my domain)
    15  O='rdennis.net'            # Organization
    16  U='Testing'                # Organizational Unit (e.g. Engineering)
    17  M='rdennis_at_rdennis.net' # Email Address of Administrator
    18
    19  if [ ! -d ${CERTDIR} ] ; then
    20      mkdir ${CERTDIR}
    21      test -d ${CERTDIR} || (echo >&2 "Error creating dir."; exit 1)
    22  fi
    23
    24  # ----------------------------------------------------------------------------
    25  # Create the server (root) key and cert:
    26
    27  # Generate key
    28  openssl req -nodes -newkey rsa:1024 -sha1 \
    29      -keyout ${CERTDIR}/rootkey.pem -out ${CERTDIR}/rootreq.pem \
    30      -subj "/C=${C}/ST=${S}/L=${L}/O=${O}/OU=${U}/CN=server/emailAddress=${M}"
    31
    32  # Self sign our cert
    33  openssl x509 -req -in ${CERTDIR}/rootreq.pem -sha1 -extensions v3_ca \
    34      -signkey ${CERTDIR}/rootkey.pem -out ${CERTDIR}/rootcert.pem
    35
    36  # Consolidate Cert and Key
    37  cat ${CERTDIR}/rootcert.pem ${CERTDIR}/rootkey.pem > ${CERTDIR}/root.pem
    38
    39  # ----------------------------------------------------------------------------
    40  # Create Client key signed w/ the root cert:
    41
    42  # Generate key
    43  openssl req -nodes -newkey rsa:1024 -sha1 \
    44      -keyout ${CERTDIR}/clientkey.pem -out ${CERTDIR}/clientreq.pem \
    45      -subj "/C=${C}/ST=${S}/L=${L}/O=${O}/OU=${U}/CN=client/emailAddress=${M}"
    46
    47  # Sign key with root cert
    48  openssl x509 -req -in ${CERTDIR}/clientreq.pem -sha1 -extensions usr_cert \
    49      -CA ${CERTDIR}/root.pem -CAkey ${CERTDIR}/root.pem -CAcreateserial    \
    50      -out ${CERTDIR}/clientcert.pem
    51
    52  # Consolidate Cert and Key
    53  cat ${CERTDIR}/clientcert.pem ${CERTDIR}/clientkey.pem \
    54      ${CERTDIR}/rootcert.pem > ${CERTDIR}/client.pem
    55
server.pl
     1  #!/usr/bin/perl
     2  # server.pl - our SSL server.  It will verify that the client's cert has been
     3  # signed by our CA, and is a valid subject.
     4
     5  use strict;
     6  use Data::Dumper;
     7  use IO::Socket::SSL;
     8
     9  # Our Valid Issuer and the Subject for our client cert
    10  my $v_issuer  = '/C=US/ST=New Jersey/L=lost/O=rdennis.net/OU=Testing/CN=server/emailAddress=rdennis@rdennis.net';
    11  my $v_subject = '/C=US/ST=New Jersey/L=lost/O=rdennis.net/OU=Testing/CN=client/emailAddress=rdennis@rdennis.net';
    12
    13  my ( $sock, $s);
    14
    15  $IO::Socket::SSL::DEBUG = 1;
    16
    17  if (
    18      !(
    19          $sock = IO::Socket::SSL->new(
    20              Listen          => 5,
    21              LocalAddr       => 'lisa',
    22              LocalPort       => 16001,
    23              Proto           => 'tcp',
    24              Reuse           => 1,
    25              SSL_verify_mode => 0x01,
    26              SSL_use_cert    => 1,
    27              SSL_ca_file     => 'certs/root.pem',
    28              SSL_cert_file   => 'certs/root.pem',
    29              SSL_key_file    => 'certs/root.pem',
    30          )
    31      )
    32    )
    33  {
    34      warn "unable to create socket: ", &IO::Socket::SSL::errstr, "\n";
    35      exit(0);
    36  }
    37  warn "socket created: $sock.\n";
    38
    39  while (1) {
    40      warn "waiting for next connection.\n";
    41
    42      while ( ( $s = $sock->accept() ) ) {
    43
    44          if ( !$s ) {
    45              warn "error: ", $sock->errstr, "\n";
    46              next;
    47          }
    48
    49          warn "connection opened ($s).\n";
    50
    51          my ($subject, $issuer);
    52          if ( ref($sock) eq "IO::Socket::SSL" ) {
    53              print $s->dump_peer_certificate();
    54              $subject = $s->peer_certificate("subject");
    55              $issuer  = $s->peer_certificate("issuer");
    56          }
    57
    58          warn "HERE";
    59          warn $s->peer_certificate('issuer');
    60
    61          warn "\t subject: '$subject'.\n";
    62          warn "\t issuer : '$issuer'.\n";
    63
    64          if (( $subject eq $v_subject ) && ( $issuer eq $v_issuer )) {
    65              my $date = localtime();
    66              print $s "VALID: my date command says it's: '$date'";
    67          }
    68
    69          close($s);
    70          warn "\t connection closed.\n";
    71      }
    72  }
    73
    74  $sock->close();
    75
    76  warn "loop exited.\n";
client.pl
     1  #!/usr/bin/perl
     2  # client.pl - our SSL Client.  Will need access to it's key and the root CA
     3
     4  use strict;
     5  use IO::Socket::SSL;
     6
     7  $IO::Socket::SSL::DEBUG = 1;
     8
     9  my $sock;
    10
    11  if (
    12      !(
    13          $sock = IO::Socket::SSL->new(
    14              PeerAddr     => 'lisa',
    15              PeerPort     => '16001',
    16              Proto        => 'tcp',
    17              SSL_use_cert => 1,
    18
    19              SSL_verify_mode => 0x06,
    20              SSL_ca_file     => 'certs/rootcert.pem',
    21              SSL_cert_file   => 'certs/clientcert.pem',
    22              SSL_key_file    => 'certs/clientkey.pem',
    23
    24          )
    25      )
    26    )
    27  {
    28      warn "unable to create socket: ", &IO::Socket::SSL::errstr, "\n";
    29      exit(0);
    30  }
    31  else {
    32      warn "connect ($sock).\n" if ($IO::Socket::SSL::DEBUG);
    33  }
    34
    35  # check server cert.
    36  my ( $subject_name, $issuer_name, $cipher );
    37  if ( ref($sock) eq "IO::Socket::SSL" ) {
    38      $subject_name = $sock->peer_certificate("subject");
    39      $issuer_name  = $sock->peer_certificate("issuer");
    40      $cipher       = $sock->get_cipher();
    41  }
    42  warn "cipher: $cipher.\n", "server cert:\n",
    43    "\t '$subject_name' \n\t '$issuer_name'.\n\n";
    44
    45  my ($buf) = $sock->getlines;
    46
    47  $sock->close();
    48
    49  print "read: '$buf'.\n";
There you go. You now have the framework needed to create a secure authenticated remote service in Perl.

VIM: My IDE

Development for me often means simply scripting in Bash, Perl, and Python. Occassionaly I will dive into C to write something. We use Python a lot at work, and so the number of internal libraries and utitlies is numerous. I recently picked up Eclipse with the pydev plugin to see if it would interested me in doing development in it. It worked fine, my only complaints were that I had to execute my scripts in Windows, and I couldn't easily change the color scheme. So I drove further into my already favorite editor: VIM. I added two plugins and changed to a darker color scheme, and voia: I'm very pleased with my VIM configuration now. So I thought I'd share what's in this recipe.

The first is the vim plugin 'supertab'. http://www.vim.org/scripts/script.php?script_id=182 This allows me to do word completion using TAB instead of Ctrl-N. All that is needed is to put the file into the ~/.vim/plugin directory.

Another plugin is the NERD_tree plugin. This places a file explorer 'toolbar' on the left hand side of the screen. http://www.vim.org/scripts/script.php?script_id=1658

I put the plugin into ~/.vim/plugin. And add the following lines of configuration into my ~/.vimrc file so that Control-t toggles the plugin.

"Toggle NERDTree

nnoremap <silent> <C-t> :NERDTreeToggle<CR>

Finally, the last plugin is taglist. http://vim.sourceforge.net/scripts/script.php?script_id=273 This requires that ctags be available. It will highlight classes, functions and global variables and you can use it to jump to the location in the file.

I just droped the plugin into ~/.vim/plugin and added the following lines to my ~/.vimrc file so that Control-y toggles the plugin.

" Toggle Tag List

let Tlist_Use_Right_Window = 1

let Tlist_Sort_Type = "name"

nnoremap <silent> <c-y> :TlistToggle<cr>

The last tweak I made to my VIM was to set the colors darker. A colleague at work forwarded me this theme. http://blog.infinitered.com/entry_files/8/ir_black.vim I made a few small modifications. Fortuantely it is very simple to understand and modify (compared with Eclipse which looked to be a lot more involved).

For this I just droped ir_black.vim into the ~/.vim/colors directory and then added the line below into my ~/.vimrc file.

colorscheme ir_black

Wednesday, December 31, 2008

NetApp Simulator

I'm working on NFSv4+KRB and came across the NetApp simulator. It's pretty easy to setup, and allows for dorking around with almost everything (clustering, iscsi, snapmirror, ...). It's helpful for someone like me who a.) first needs to get acquainted with managing a netapp filler without breaking anything, and b.) wants to test filer kerberos configuration. (Ultimately I'll need a physical filer for further performance and stability testing, but it's a handy start.)

To read about what the simulator is, what it's limitations, strenghts, and requirements are, check out this: http://www.unitek.com/training/netapp/blog/2008/05/strengths-and-weaknesses-of-your-netapp-simulator/

Other useful links:

Download Simulator: (NOW account required)
http://now.netapp.com/NOW/cgi-bin/simulator

A tutorial series on using the Netapp Simulator:
http://www.unitek.com/training/netapp/blog/category/uncategorized/page/2/

Netapp Simulator 7.3 -- Installing the Simulator
http://tendam.files.wordpress.com/2008/12/simulator7_3.pdf

Netapp Simulator Forum:
http://communities.netapp.com/community/technet/interfaces_and_tools/simulator?view=discussions

Useful Netapp Commands:
http://unixfoo.blogspot.com/2007/10/untitled.html

Netapp Snapvault guide:
http://www.geekyfacts.com/index.php?option=com_content&view=article&id=64:snapvault&catid=38:netapp&Itemid=58

Netapp ONTAP SDK:
http://communities.netapp.com/docs/DOC-1110

Netapp Technical Reports:
http://www.netapp.com/us/library/technical-reports/15619132.html?&letter=&sortBy=partNumber&pageSize=400

Learning Oracle RAC with the NetApp Simulator:
http://www.oracle.com/technology/pub/articles/garg-netapp-simulator.html

Tuesday, December 30, 2008

Staring Over

So I'm staring this blog thing up again for three reasons. The first is I'd like to get better at technical writing and I'm hoping this helps. And the second is that sometimes I'll have a friend/colleague ask me some off topic question, and I thought rather than email back this might be a good way to answer. Finally, there are times where I forget why or how I did something and I figure this will be a better place than any to keep a record for myself.