Perl: useful tidbits from work

For a few years now, a large part of my job has depended on my writing and maintaining Perl scripts. I thought it would be wise, for my own sake, to note down some of the things I’ve learned along the way. I don’t claim these notes to contain optimal solutions and look forward to any comments they may spark.

Adding an “include” directory to a file

I wasn’t the one to come up with this exact format, and I feel like some of it is extra, however, it has worked and was proliferated throughout many files before I picked them up, so once I have the need to use it for myself, I’ll take the opportunity to figure out what’s really needed. It does show the flexibility of what can be added in the BEGIN block as is.

    use File::Spec::Functions 'catfile';
    use File::Basename 'dirname';
    push @INC, catfile(dirname($0), '../other_location/Perl_modules');

Generic Code, Project Specifics in a Hash

We had multiple projects which were very similar in what needed to be done via the Perl scripts. To keep the code from being hardcoded to a project, we used a Perl module Custom::Project, containing a hash with details about the products and their dependencies. For instance, one layout might be:

package Custom::Project;
  %Products = (
    SourceA_Product => {
      path        => "source_a",
      label_name  => "MYSOURCEA",
    SourceB_Product => {
      path        => "source_b",
      label_name  => "MYSOURCEB",
    Top_Product => {
      path        => "top",
      label_name  => "TOP",
      subproducts => ["SourceA_Product", "SourceB_Product"],

With access to this hash, if I want to apply a label, I can grab the information as long as I know what product I need the information for using something like $Custom::Project::Products{“SourceA_Product”}{label_name}. Or if I’d like to cycle through the subproducts of Top_Product, I can use the array directly by using @{$Custom::Project::Products{“Top_Product”}{subproducts}}

Precompile a Regex

Sometimes it can be useful to store a regex in a variable. In one instance I setup a regex in a variable because it had changed a couple times already, and it was part of a few steps of substitutions, so I wanted to pull out what I could. What I did was create a variable

my $to_be_replaced = qr/NUMBER_(\d)_REPLACEABLE/;

Then, later, in the substitution phase, I was able to do something like this:

$file_contents =~ s/$to_be_replaced/New_Name_$1/g;

Important here is that the number in the text to be replaced was captured and used in the replacement text.

Name both key and value when iterating over a hash

Instead of foreach and only naming a key, an alternative is to use

while ((my $key, my $value) = each %hash) {}

Passing values to subroutines

Previously, I had always used shift to pop the values passed to subroutines. There is an alternative which looks something like this

my ($value1, $value2, $value3) = @_;

Since Perl doesn’t require you to be strict about dictating what will be passed where, this makes a simple way to quickly write out the arguments passed through, though does require attention be paid when making changes. However, I feel it is more readable than using repetitive shifts.
A second point is not really specific to argument passing, but is useful to the cause. Obviously you cannot pass an array to a subroutine directly as the way the arguments are passed is fairly blatantly like an array based on my above note. Passing an array can be critical, though, and it’s not out of the question, it’s simply a matter of how you pass it. Instead of pushing the whole array in by passing it as @my_array, you can pass a reference to the array using \@my_array. Once the subroutine receives it, it will need to dereference what it receives, we’ll call it $my_array_ref in the subroutine, as such: @{$my_array_ref}. Note that this goes for returning information from a subroutine as well; you can return multiple parameters, and they can be references.

Execution Calls

There are a few different ways to execute something from within a Perl script. They all return slightly differently, meaning it’s important to choose the right one.

system($CMD); # Returns success or failure result
`$CMD`;       # Returns output of running command (the characters are backticks and this is effectively the same as qx/$CMD/; & qx{$CMD};)
exec($CMD);   # Does not return

I know there is also an option open(), but I’m not as familiar with it in terms of executing commands, instead of things like opening files. I will direct you to Perl HowTo if you wish for more detailed descriptions.

Saving a file with Unix newlines

This may not be an issue if you’re on a Unix or Linux machine, but if you’re on a Windows machine and the file you’re creating needs to have the Unix newline format, there’s an one extra step you can take to make it happen.

my $result = open(FILE, ">" . $filename) ? 0 : 1;
binmode FILE;

Parallel tasks

Sometimes it’s possible to perform tasks in parallel, and Perl has a few options for performing parallel tasks, but I’ve started to prefer threads (& threads::shared).

use threads;
use threads::shared;
my @threads;
foreach my $folder (@list)
  my $thr = threads->new(\&MyFunction, $arg1, $arg2, $arg3);
  push(@threads, $thr);
foreach (@threads)
  $result = $_->join;
  die "Failed with result $result\n" if ($result);
undef @threads;

I did at one point run across a problem due to the size of what was being built by several threads, but I was able to alleviate the problem by increase the stack size by replacing use threads; with:

use threads ('stack_size' => 4096 * 10);

Here 4096 is the page size, and I can’t recall if I ever determined the default stack size, but this size seemed to be sufficient.

Dialog Boxes

Although it’s nice to fully automate tasks, sometimes, there are things you cannot escape requiring the user to do. In one such case, I decided that command line prompting would not be sufficient, so I opted to present the user with a dialog box.

use Tk;
use Tk::DialogBox;
do {
  my $mw = MainWindow->new;
  $dialog_response = $mw->messageBox(-title=>"My Dialog Box Title",
		  -message=>"Please answer Yes or No to my question here",
} while ($dialog_response ne "Yes");

ClearCase: useful tidbits from work

Where I work, the primary version control system used is ClearCase. It’s the first version control system I learned, so even through its challenges, it remains my reference point for learning other version control systems. It’s easy to become familiar with the basics, but I’ve learned a few tricks that I find myself having to re-look-up, hoping I can find them again because I don’t need them frequently. That and, a lot of the tasks we used to do manually, we have rolled into scripts which govern building tasks so less is left to human error. So, here I will note some of the things I’ve found useful, but don’t always remember without looking them up.
Two of my favorite references online are:
Phil for Humanity (all the articles labeled “ClearCase Support”)
Then you have the regular, old manual:
IBM Cleartool Manual

cleartool ls

We’ll start with something obvious, that I don’t really have to look up, I just tend to have to think for a few seconds before I remember that it will help me figure out why I’m looking at the wrong thing. To clarify, if you’re in a directory, and you use cleartool ls, it will list the contents of the directory and which configuration specification rule is active on said item.

cleartool lsco

This, I found, was important for avoiding trouble when importing. lsco (or lscheckout), as may seem obvious, lists files checkedout. What was useful to me was checking on a specific branch, so I was using something like:

cleartool lsco -brtype BRANCH -recurse

Another useful checkout search is for all your own checkouts (via -me) in your view (current view via -cview), in all vobs (via -avobs)

cleartool lsco -me -avobs -cview

Find a branch or label

Need to know what labels or branches exist?

cleartool lstype -kind brtype -invob \my_vob

The -invob option can be super helpful if you are potentially in a different vob.
Alternatively, if you want to check if a single label or branch exists, another option is to ask for its description.

cleartool desc -s lbtype:MY_LABEL


We regularly imported source from a vendor, which we added on to, so it was important to keep things organized. This was our standard import call.

clearfsimport -nsetevent -recurse -mklabel <LABEL> <DIR_TO_IMPORT> <CC_LOCATION_TO_IMPORT_TO> 1> Logfileofyourchoice.txt 2>Errorlogfileofyourchoice.txt

Finding things

Just as an example, this will find all items on my_branch and as it finds each one, it will bring up a version tree for it. This can be useful in small cases, but is not advised when there will be many results.

cleartool find . -version "version(.../my_branch/LATEST)" -exe "cleartool lsvt -g %CLEARCASE_PN%"

(Note: the use of “.” is to indicate that the search should be in the current folder; you may replace this with a different path.)
Of course, alternatively, you can get just a list of all the files by replacing -exe and its parameters with simply -print. It can also be helpful sometimes to find files changed between two points, in which case, in the -version parameter block, adding && !version(.../other_branch/LATEST) is helpful. Note that -version isn’t terribly picky, so you can use branches or labels in the ().

Doing something with what you find

In the previous example, the -exe told ClearCase to open the version tree for each file found. Another example is to label each file found:

cleartool find . -all -version "lbtype(<LABEL>)" -exe "ct mklabel <OTHER_LABEL> %CLEARCASE_XPN%"

Here we use the clearcase special variable (with the extended path name, which would be more important if we changed “.” to a different folder) to indicate the file we’d like to apply a secondary label to.

BeagleBone first troubles & resolutions

So, for Fall semester 2012, I’m taking a class focusing on embedded Linux systems using the BeagleBone. I thought it would be worthwhile to jot down the issues I come across and the solutions I find.
I have an A6 revision BeagleBone. I purchased the “kit” sold at Adafruit because I’m always excited to have extras for the projects I imagine I’ll get around to at some point. :) Also, the kit comes with a power supply, which I wanted to have simply because I didn’t want to leave a computer running simply to power the BeagleBone (seems kind of silly to power a low power device with a high power one, especially when I’d like to have it accessible all the time).

First things first, I perused the Adafruit tutorial and the GigaMegaBlog setup post.

First trouble: Installing BeagleBone Drivers on Windows fails!

I wanted to install the drivers on both a Windows Vista Enterprise 32 bit computer and a Windows 7 64 bit computer. Neither worked. After some searching, the best hint I had was to do with the PID (Product ID), but it took me a while longer to sort out a solution. Here’s what I came up with (as copied from a post to the class messageboard):

The easiest place to get the complete set of driver files is from the mass storage the BeagleBone presents itself as when first connected. Make a copy of the BB:\Drivers\Windows folder on your host machine, wherever you like to organize such things.
Next, open the two .inf files found in Windows\src\files\FTDI\ and do a find and replace: A6D0 -> 6010. A6D0 is the PID for BeagleBone revisions A4 and older, 6010 is the PID for BeagleBone revisions A5 and newer.

Next, make sure to uninstall any drivers you tried already to install. I would also suggest disconnecting the BeagleBone at this point.

The most sure fire way from here, I think, to get the drivers to connect correctly, is to explicitly use the “have disk” option to install. Once you’ve reconnected the BeagleBone, ydevice and select “Update Driver”, then “Browse”, then “Let me pick”, then just skip the selection, then “I have a disk”, then select the modified .inf files in Windows\src\files\FTDI\. I’m not completely sure if there’s any requirement that you select them in specific, but I used ftdiport.inf for USB Serial port, and ftdibus.inf for XDS100v2 compatible USB Serial Converter A & B.

Hopefully this will work for you if you’re trying to get the drivers working with Windows!

Second trouble: Cannot SSH into BeagleBone after restarting!

Having played around, setting up dynamic DNS on my router and checking things out via SSH several times, there was some reason I decided I needed to restart (shutdown -r now). Later, I decided to check in on my BeagleBone and couldn’t reach it! I hooked up the trusty USB cable and poked around, coming up with nothing. A thought occurred to me, and I decided to try to restart DropBear (the SSH service). Lo and behold, it said it found no instances to restart (but claimed to restart anyway). This did not start DropBear, somewhat to my surprise, so I took a guess and tried starting DropBear via:

/etc/init.d/dropbear start

It worked! I haven’t restarted since, so I haven’t sorted out whether it was a fluke that it didn’t start, and I haven’t found anything specific in my searching about why this might have happened. Perhaps an update to this post will be forthcoming.