View Source: perldsc(1) - Waikato Linux Users Group

Edit PageHistory Diff Info LikePages
PERLDSC
!!!PERLDSC
NAME
DESCRIPTION
REFERENCES
COMMON MISTAKES
CAVEAT ON PRECEDENCE
WHY YOU SHOULD ALWAYS use strict
DEBUGGING
CODE EXAMPLES
ARRAYS OF ARRAYS
HASHES OF ARRAYS
ARRAYS OF HASHES
HASHES OF HASHES
MORE ELABORATE RECORDS
Database Ties
SEE ALSO
AUTHOR
----
!!NAME


perldsc - Perl Data Structures Cookbook
!!DESCRIPTION


The single feature most sorely lacking in the Perl
programming language prior to its 5.0 release was complex
data structures. Even without direct language support, some
valiant programmers did manage to emulate them, but it was
hard work and not for the faint of heart. You could
occasionally get away with the $m{$AoA,$b} notation
borrowed from __awk__ in which the keys are actually more
like a single concatenated string
, but traversal and sorting were
difficult. More desperate programmers even hacked Perl's
internal symbol table directly, a strategy that proved hard
to develop and maintain--to put it mildly.


The 5.0 release of Perl let us have complex data structures.
You may now write something like this and all of a sudden,
you'd have a array with three dimensions!


    for $x (1 .. 10) {
for $y (1 .. 10) {
for $z (1 .. 10) {
$AoA[[$x][[$y][[$z] =
$x ** $y + $z;
}
}
}
Alas, however simple this may appear, underneath it's a much more elaborate construct than meets the eye!


How do you print it out? Why can't you say just print
@AoA? How do you sort it? How can you pass it to a
function or get one of these back from a function? Is is an
object? Can you save it to disk to read back later? How do
you access whole rows or columns of that matrix? Do all the
values have to be numeric?


As you see, it's quite easy to become confused. While some
small portion of the blame for this can be attributed to the
reference-based implementation, it's really more due to a
lack of existing documentation with examples designed for
the beginner.


This document is meant to be a detailed but understandable
treatment of the many different sorts of data structures you
might want to develop. It should also serve as a cookbook of
examples. That way, when you need to create one of these
complex data structures, you can just pinch, pilfer, or
purloin a drop-in example from here.


Let's look at each of these possible constructs in detail.
There are separate sections on each of the
following:


arrays of arrays


hashes of arrays


arrays of hashes


hashes of hashes


more elaborate constructs


But for now, let's look at general issues common to all
these types of data structures.
!!REFERENCES


The most important thing to understand about all data
structures in Perl -- including multidimensional arrays--is
that even though they might appear otherwise, Perl
@ARRAYs and %HASHes are all internally
one-dimensional. They can hold only scalar values (meaning a
string, number, or a reference). They cannot directly
contain other arrays or hashes, but instead contain
''references'' to other arrays or hashes.


You can't use a reference to a array or hash in quite the
same way that you would a real array or hash. For C or C
++ programmers unused to distinguishing
between arrays and pointers to the same, this can be
confusing. If so, just think of it as the difference between
a structure and a pointer to a structure.


You can (and should) read more about references in the
perlref(1) man page. Briefly, references are rather
like pointers that know what they point to. (Objects are
also a kind of reference, but we won't be needing them right
away--if ever.) This means that when you have something
which looks to you like an access to a
two-or-more-dimensional array and/or hash, what's really
going on is that the base type is merely a one-dimensional
entity that contains references to the next level. It's just
that you can ''use'' it as though it were a
two-dimensional one. This is actually the way almost all C
multidimensional arrays work as well.


    $array[[7][[12]                       # array of arrays
$array[[7]{string}                   # array of hashes
$hash{string}[[7]                    # hash of arrays
$hash{string}{'another string'}     # hash of hashes
Now, because the top level contains only references, if you try to print out your array in with a simple ''print()'' function, you'll get something that doesn't look very nice, like this:


    @AoA = ( [[2, 3], [[4, 5, 7], [[0] );
print $AoA[[1][[2];
7
print @AoA;
ARRAY(0x83c38)ARRAY(0x8b194)ARRAY(0x8b1d0)
That's because Perl doesn't (ever) implicitly dereference your variables. If you want to get at the thing a reference is referring to, then you have to do this yourself using either prefix typing indicators, like ${$blah}, @{$blah}, @{$blah[[$i]}, or else postfix pointer arrows, like $a-, $h-, or even $ob-.
!!COMMON MISTAKES


The two most common mistakes made in constructing something
like an array of arrays is either accidentally counting the
number of elements or else taking a reference to the same
memory location repeatedly. Here's the case where you just
get the count instead of a nested array:


    for $i (1..10) {
@array = somefunc($i);
$AoA[[$i] = @array;      # WRONG!
}
That's just the simple case of assigning an array to a scalar and getting its element count. If that's what you really and truly want, then you might do well to consider being a tad more explicit about it, like this:


    for $i (1..10) {
@array = somefunc($i);
$counts[[$i] = scalar @array;
}
Here's the case of taking a reference to the same memory location again and again:


    for $i (1..10) {
@array = somefunc($i);
$AoA[[$i] = @array;     # WRONG!
}
So, what's the big problem with that? It looks right, doesn't it? After all, I just told you that you need an array of references, so by golly, you've made me one!


Unfortunately, while this is true, it's still broken. All
the references in @AoA refer to the ''very same
place'', and they will therefore all hold whatever was
last in @array! It's similar to the problem
demonstrated in the following C program:


    #include
        printf(
Which will print


    daemon name is daemon
root name is daemon
The problem is that both rp and dp are pointers to the same location in memory! In C, you'd have to remember to ''malloc()'' yourself some new memory. In Perl, you'll want to use the array constructor [[] or the hash constructor {} instead. Here's the right way to do the preceding broken code fragments:


    for $i (1..10) {
@array = somefunc($i);
$AoA[[$i] = [[ @array ];
}
The square brackets make a reference to a new array with a ''copy'' of what's in @array at the time of the assignment. This is what you want.


Note that this will produce something similar, but it's much
harder to read:


    for $i (1..10) {
@array = 0 .. $i;
@{$AoA[[$i]} = @array;
}
Is it the same? Well, maybe so--and maybe not. The subtle difference is that when you assign something in square brackets, you know for sure it's always a brand new reference with a new ''copy'' of the data. Something else could be going on in this new case with the @{$AoA[[$i]}} dereference on the left-hand-side of the assignment. It all depends on whether $AoA[[$i] had been undefined to start with, or whether it already contained a reference. If you had already populated @AoA with references, as in


    $AoA[[3] = @another_array;
Then the assignment with the indirection on the left-hand-side would use the existing reference that was already there:


    @{$AoA[[3]} = @array;
Of course, this ''would'' have the ``interesting'' effect of clobbering @another_array. (Have you ever noticed how when a programmer says something is ``interesting'', that rather than meaning ``intriguing'', they're disturbingly more apt to mean that it's ``annoying'', ``difficult'', or both? :-)


So just remember always to use the array or hash
constructors with [[] or {}, and you'll be
fine, although it's not always optimally
efficient.


Surprisingly, the following dangerous-looking construct will
actually work out fine:


    for $i (1..10) {
my @array = somefunc($i);
$AoA[[$i] = @array;
}
That's because ''my()'' is more of a run-time statement than it is a compile-time declaration ''per se''. This means that the ''my()'' variable is remade afresh each time through the loop. So even though it ''looks'' as though you stored the same variable reference each time, you actually did not! This is a subtle distinction that can produce more efficient code at the risk of misleading all but the most experienced of programmers. So I usually advise against teaching it to beginners. In fact, except for passing arguments to functions, I seldom like to see the gimme-a-reference operator (backslash) used much at all in code. Instead, I advise beginners that they (and most of the rest of us) should try to use the much more easily understood constructors [[] and {} instead of relying upon lexical (or dynamic) scoping and hidden reference-counting to do the right thing behind the scenes.


In summary:


    $AoA[[$i] = [[ @array ];      # usually best
$AoA[[$i] = @array;         # perilous; just how my() was that array?
@{ $AoA[[$i] } = @array;     # way too tricky for most programmers
!!CAVEAT ON PRECEDENCE


Speaking of things like @{$AoA[[$i]}, the following
are actually the same thing:


    $aref-
That's because Perl's precedence rules on its five prefix dereferencers (which look like someone swearing: $ @ * % ) make them bind more tightly than the postfix subscripting brackets or braces! This will no doubt come as a great shock to the C or C ++ programmer, who is quite accustomed to using *a[[i] to mean what's pointed to by the ''i'th'' element of a. That is, they first take the subscript, and only then dereference the thing at that subscript. That's fine in C, but this isn't C.


The seemingly equivalent construct in Perl,
$$aref[[$i] first does the deref of $aref,
making it take $aref as a reference to an array,
and then dereference that, and finally tell you the
''i'th'' value of the array pointed to by $AoA.
If you wanted the C notion, you'd have to write
${$AoA[[$i]} to force the $AoA[[$i] to get
evaluated first before the leading $
dereferencer.
!!WHY YOU SHOULD ALWAYS use strict


If this is starting to sound scarier than it's worth, relax.
Perl has some features to help you avoid its most common
pitfalls. The best way to avoid getting confused is to start
every program like this:


    #!/usr/bin/perl -w
use strict;
This way, you'll be forced to declare all your variables with ''my()'' and also disallow accidental ``symbolic dereferencing''. Therefore if you'd done this:


    my $aref = [[
[[
    print $aref[[2][[2];
The compiler would immediately flag that as an error ''at compile time'', because you were accidentally accessing @aref, an undeclared variable, and it would thereby remind you to write instead:


    print $aref-
!!DEBUGGING


Before version 5.002, the standard Perl debugger didn't do a
very nice job of printing out complex data structures. With
5.002 or above, the debugger includes several new features,
including command line editing as well as the x
command to dump out complex data structures. For example,
given the assignment to $AoA above, here's the
debugger output:


    DB
!!CODE EXAMPLES


Presented with little comment (these will get their own
manpages someday) here are short code examples illustrating
access of various types of data structures.
!!ARRAYS OF ARRAYS


__Declaration of a ARRAY OF
ARRAYS__


 @AoA = (
[[


__Generation of a ARRAY OF
ARRAYS__


 # reading from file
while (
 # calling a function
for $i ( 1 .. 10 ) {
$AoA[[$i] = [[ somefunc($i) ];
}
 # using temp vars
for $i ( 1 .. 10 ) {
@tmp = somefunc($i);
$AoA[[$i] = [[ @tmp ];
}
 # add to an existing row
push @{ $AoA[[0] },


__Access and Printing of a ARRAY OF
ARRAYS__


 # one element
$AoA[[0][[0] =
 # another element
$AoA[[1][[1] =~ s/(w)/u$1/;
 # print the whole thing with refs
for $aref ( @AoA ) {
print
 # print the whole thing with indices
for $i ( 0 .. $#AoA ) {
print
 # print the whole thing one at a time
for $i ( 0 .. $#AoA ) {
for $j ( 0 .. $#{ $AoA[[$i] } ) {
print
!!HASHES OF ARRAYS


__Declaration of a HASH OF
ARRAYS__


 %HoA = (
flintstones        =


__Generation of a HASH OF
ARRAYS__


 # reading from file
# flintstones: fred barney wilma dino
while (
 # reading from file; more temps
# flintstones: fred barney wilma dino
while ( $line =
 # calling a function that returns a list
for $group (
 # likewise, but using temps
for $group (
 # append new members to an existing family
push @{ $HoA{


__Access and Printing of a HASH OF
ARRAYS__


 # one element
$HoA{flintstones}[[0] =
 # another element
$HoA{simpsons}[[1] =~ s/(w)/u$1/;
 # print the whole thing
foreach $family ( keys %HoA ) {
print
 # print the whole thing with indices
foreach $family ( keys %HoA ) {
print
 # print the whole thing sorted by number of members
foreach $family ( sort { @{$HoA{$b}}
 # print the whole thing sorted by number of members and name
foreach $family ( sort {
@{$HoA{$b}}
!!ARRAYS OF HASHES


__Declaration of a ARRAY OF
HASHES__


 @AoH = (
{
Lead     =


__Generation of a ARRAY OF
HASHES__


 # reading from file
# format: LEAD=fred FRIEND=barney
while (
 # reading from file
# format: LEAD=fred FRIEND=barney
# no temp
while (
 # calling a function  that returns a key/value pair list, like
#
 # likewise, but using no temp vars
while (
 # add key/value to an element
$AoH[[0]{pet} =


__Access and Printing of a ARRAY OF
HASHES__


 # one element
$AoH[[0]{lead} =
 # another element
$AoH[[1]{lead} =~ s/(w)/u$1/;
 # print the whole thing with refs
for $href ( @AoH ) {
print
 # print the whole thing with indices
for $i ( 0 .. $#AoH ) {
print
 # print the whole thing one at a time
for $i ( 0 .. $#AoH ) {
for $role ( keys %{ $AoH[[$i] } ) {
print
!!HASHES OF HASHES


__Declaration of a HASH OF
HASHES__


 %HoH = (
flintstones =


__Generation of a HASH OF
HASHES__


 # reading from file
# flintstones: lead=fred pal=barney wife=wilma pet=dino
while (
 # reading from file; more temps
while (
 # calling a function  that returns a key,value hash
for $group (
 # likewise, but using temps
for $group (
 # append new members to an existing family
%new_folks = (
wife =
 for $what (keys %new_folks) {
$HoH{flintstones}{$what} = $new_folks{$what};
}


__Access and Printing of a HASH OF
HASHES__


 # one element
$HoH{flintstones}{wife} =
 # another element
$HoH{simpsons}{lead} =~ s/(w)/u$1/;
 # print the whole thing
foreach $family ( keys %HoH ) {
print
 # print the whole thing  somewhat sorted
foreach $family ( sort keys %HoH ) {
print
 # print the whole thing sorted by number of members
foreach $family ( sort { keys %{$HoH{$b}}
 # establish a sort order (rank) for each role
$i = 0;
for ( qw(lead wife son daughter pal pet) ) { $rank{$_} = ++$i }
 # now print the whole thing sorted by number of members
foreach $family ( sort { keys %{ $HoH{$b} }
!!MORE ELABORATE RECORDS


__Declaration of MORE ELABORATE
RECORDS__


Here's a sample showing how to create and use a record whose
fields are of many different sorts:


     $rec = {
TEXT      =
     print $rec-
     print $rec-
     print $rec-
     $answer = $rec-
     # careful of extra block braces on fh ref
print { $rec-
     use !FileHandle;
$rec-


__Declaration of a HASH OF COMPLEX
RECORDS__


     %TV = (
flintstones =
        jetsons     =
        simpsons    =


__Generation of a HASH OF COMPLEX
RECORDS__


     # reading from file
# this is most easily done by having the file itself be
# in the raw data format as shown above.  perl is happy
# to parse complex data structures if declared as data, so
# sometimes it's easiest to do that
     # here's a piece by piece build up
$rec = {};
$rec-
     @members = ();
# assume this file in field=value syntax
while (
     # now remember the whole thing
$TV{ $rec-
     ###########################################################
# now, you might want to make interesting extra fields that
# include pointers back into the same data structure so if
# change one piece, it changes everywhere, like for example
# if you wanted a {kids} field that was a reference
# to an array of the kids' records without having duplicate
# records and thus update problems.
###########################################################
foreach $family (keys %TV) {
$rec = $TV{$family}; # temp pointer
@kids = ();
for $person ( @{ $rec-
     # you copied the array, but the array itself contains pointers
# to uncopied objects. this means that if you make bart get
# older via
     $TV{simpsons}{kids}[[0]{age}++;
     # then this would also change in
print $TV{simpsons}{members}[[2]{age};
     # because $TV{simpsons}{kids}[[0] and $TV{simpsons}{members}[[2]
# both point to the same underlying anonymous hash table
     # print the whole thing
foreach $family ( keys %TV ) {
print
!!Database Ties


You cannot easily tie a multilevel data structure (such as a
hash of hashes) to a dbm file. The first problem is that all
but GDBM and Berkeley DB have
size limitations, but beyond that, you also have problems
with how references are to be represented on disk. One
experimental module that does partially attempt to address
this need is the MLDBM module. Check your
nearest CPAN site as described in perlmodlib
for source code to MLDBM .
!!SEE ALSO


perlref(1), perllol(1), perldata(1),
perlobj(1)
!!AUTHOR


Tom Christiansen
tchrist@perl.com''''


Last update: Wed Oct 23 04:57:50 MET DST
1996
----
2 pages link to perldsc(1):
This page is a man page (or other imported legacy content). We are unable to automatically determine the license status of this page.
Last edited on Tuesday, June 4, 2002 12:22:33 am by "perry"
Edit PageHistory Diff Info LikePages