Penguin
Recent Changes
Blame: perlxstut(1)
EditPageHistoryDiffInfoLikePages
Annotated edit history of perlxstut(1) version 2, including all changes. View license author blame.
Rev Author # Line
1 perry 1 PERLXSTUT
2 !!!PERLXSTUT
3 NAME
4 DESCRIPTION
5 SPECIAL NOTES
6 TUTORIAL
7 See also
8 Author
9 ----
10 !!NAME
11
12
13 perlXStut - Tutorial for writing XSUBs
14 !!DESCRIPTION
15
16
17 This tutorial will educate the reader on the steps involved
18 in creating a Perl extension. The reader is assumed to have
19 access to perlguts, perlapi and perlxs.
20
21
22 This tutorial starts with very simple examples and becomes
23 more complex, with each new example adding new features.
24 Certain concepts may not be completely explained until later
25 in the tutorial in order to slowly ease the reader into
26 building extensions.
27
28
29 This tutorial was written from a Unix point of view. Where I
30 know them to be otherwise different for other platforms
31 (e.g. Win32), I will list them. If you find something that
32 was missed, please let me know.
33 !!SPECIAL NOTES
34
35
36 __make__
37
38
39 This tutorial assumes that the make program that Perl is
40 configured to use is called make. Instead of
41 running ``make'' in the examples that follow, you may have
42 to substitute whatever make program Perl has been configured
43 to use. Running __perl -V:make__ should tell you what it
44 is.
45
46
47 __Version caveat__
48
49
50 When writing a Perl extension for general consumption, one
51 should expect that the extension will be used with versions
52 of Perl different from the version available on your
53 machine. Since you are reading this document, the version of
54 Perl on your machine is probably 5.005 or later, but the
55 users of your extension may have more ancient
56 versions.
57
58
59 To understand what kinds of incompatibilities one may
60 expect, and in the rare case that the version of Perl on
61 your machine is older than this document, see the section on
62 ``Troubleshooting these Examples'' for more
63 information.
64
65
66 If your extension uses some features of Perl which are not
67 available on older releases of Perl, your users would
68 appreciate an early meaningful warning. You would probably
69 put this information into the ''README''
70 file, but nowadays installation of extensions may be
71 performed automatically, guided by ''CPAN
72 .pm'' module or other tools.
73
74
2 perry 75 In !MakeMaker-based installations, ''Makefile.PL''
1 perry 76 provides the earliest opportunity to perform version checks.
77 One can put something like this in ''Makefile.PL'' for
78 this purpose:
79
80
81 eval { require 5.007 }
82 or die
83
84
85 __Dynamic Loading versus Static Loading__
86
87
88 It is commonly thought that if a system does not have the
89 capability to dynamically load a library, you cannot build
90 XSUBs. This is incorrect. You ''can'' build them, but you
91 must link the XSUBs subroutines with the rest of Perl,
92 creating a new executable. This situation is similar to Perl
93 4.
94
95
96 This tutorial can still be used on such a system. The
97 XSUB build mechanism will check the system
98 and build a dynamically-loadable library if possible, or
99 else a static library and then, optionally, a new
100 statically-linked executable with that static library linked
101 in.
102
103
104 Should you wish to build a statically-linked executable on a
105 system which can dynamically load libraries, you may, in all
106 the following examples, where the command
107 make`` with no arguments is executed, run the
108 command ''make perl
109
110
111 If you have generated such a statically-linked executable by
112 choice, then instead of saying make test``,
113 you should say ''make test_static``. On systems
114 that cannot build dynamically-loadable libraries at all,
115 simply saying ''make test
116 !!TUTORIAL
117
118
119 Now let's go on with the show!
120
121
122 __EXAMPLE 1__
123
124
125 Our first extension will be very simple. When we call the
126 routine in the extension, it will print out a well-known
127 message and return.
128
129
130 Run h2xs -A -n Mytest
131 MANIFEST , Makefile.PL, Mytest.pm, Mytest.xs,
132 test.pl, and Changes.
133
134
135 The MANIFEST file contains the names of all
136 the files just created in the Mytest directory.
137
138
139 The file Makefile.PL should look something like
140 this:
141
142
2 perry 143 use !ExtUtils::!MakeMaker;
144 # See lib/!ExtUtils/!MakeMaker.pm for details of how to influence
1 perry 145 # the contents of the Makefile that is written.
2 perry 146 !WriteMakefile(
1 perry 147 NAME =
148 The file Mytest.pm should start with something like this:
149
150
151 package Mytest;
152 use strict;
153 use warnings;
154 require Exporter;
2 perry 155 require !DynaLoader;
156 our @ISA = qw(Exporter !DynaLoader);
1 perry 157 # Items to export into callers namespace by default. Note: do not export
158 # names by default without a very good reason. Use EXPORT_OK instead.
159 # Do not simply export all your public functions/methods/constants.
160 our @EXPORT = qw(
161 );
162 our $VERSION = '0.01';
163 bootstrap Mytest $VERSION;
164 # Preloaded methods go here.
165 # Autoload methods go after __END__, and are processed by the autosplit program.
166 1;
167 __END__
168 # Below is the stub of documentation for your module. You better edit it!
169 The rest of the .pm file contains sample code for providing documentation for the extension.
170
171
172 Finally, the Mytest.xs file should look something like
173 this:
174
175
176 #include
177 MODULE = Mytest PACKAGE = Mytest
178 Let's edit the .xs file by adding this to the end of the file:
179
180
181 void
182 hello()
183 CODE:
184 printf(
185 It is okay for the lines starting at the `` CODE: '' line to not be indented. However, for readability purposes, it is suggested that you indent CODE: one level and the lines following one more level.
186
187
188 Now we'll run perl Makefile.PL
189
190
191 % perl Makefile.PL
192 Checking if your kit is complete...
193 Looks good
194 Writing Makefile for Mytest
195 %
196 Now, running make will produce output that looks something like this (some long lines have been shortened for clarity and some extraneous lines have been deleted):
197
198
199 % make
200 umask 0
201 You can safely ignore the line about ``prototyping behavior'' - it is explained in the section ``The PROTOTYPES: Keyword'' in perlxs.
202
203
204 If you are on a Win32 system, and the build process fails
205 with linker errors for functions in the C library, check if
206 your Perl is configured to use PerlCRT (running __perl
207 -V:libc__ should show you if this is the case). If Perl is
208 configured to use PerlCRT, you have to make sure PerlCRT.lib
209 is copied to the same location that msvcrt.lib lives in, so
210 that the compiler can find it on its own. msvcrt.lib is
211 usually found in the Visual C compiler's lib directory (e.g.
2 perry 212 C:/!DevStudio/VC/lib).
1 perry 213
214
215 Perl has its own special way of easily writing test scripts,
216 but for this example only, we'll create our own test script.
217 Create a file called hello that looks like
218 this:
219
220
221 #! /opt/perl5/bin/perl
2 perry 222 use !ExtUtils::testlib;
1 perry 223 use Mytest;
224 Mytest::hello();
225 Now we make the script executable (chmod -x hello), run the script and we should see the following output:
226
227
228 % ./hello
229 Hello, world!
230 %
231
232
233 __EXAMPLE 2__
234
235
236 Now let's add to our extension a subroutine that will take a
237 single numeric argument as input and return 0 if the number
238 is even or 1 if the number is odd.
239
240
241 Add the following to the end of Mytest.xs:
242
243
244 int
245 is_even(input)
246 int input
247 CODE:
248 RETVAL = (input % 2 == 0);
249 OUTPUT:
250 RETVAL
251 There does not need to be white space at the start of the int input`` line, but it is useful for improving readability. Placing a semi-colon at the end of that line is also optional. Any amount and kind of white space may be placed between the ''int`` and ''input
252
253
254 Now re-run make to rebuild our new shared
255 library.
256
257
258 Now perform the same steps as before, generating a Makefile
259 from the Makefile.PL file, and running make.
260
261
262 In order to test that our extension works, we now need to
263 look at the file test.pl. This file is set up to imitate the
264 same kind of testing structure that Perl itself has. Within
265 the test script, you perform a number of tests to confirm
266 the behavior of the extension, printing ``ok'' when the test
267 is correct, ``not ok'' when it is not. Change the print
268 statement in the BEGIN block to print
269 ``1..4'', and add the following code to the end of the
270 file:
271
272
273 print
274 We will be calling the test script through the command make test
275
276
277 % make test
278 PERL_DL_NONLAZY=1 /opt/perl5.004/bin/perl (lots of -I arguments) test.pl
279 1..4
280 ok 1
281 ok 2
282 ok 3
283 ok 4
284 %
285
286
287 __What has gone on?__
288
289
290 The program h2xs is the starting point for creating
291 extensions. In later examples we'll see how we can use h2xs
292 to read header files and generate templates to connect to C
293 routines.
294
295
296 h2xs creates a number of files in the extension directory.
297 The file Makefile.PL is a perl script which will generate a
298 true Makefile to build the extension. We'll take a closer
299 look at it later.
300
301
302 The .pm and .xs files contain the meat of the extension. The
303 .xs file holds the C routines that make up the extension.
304 The .pm file contains routines that tell Perl how to load
305 your extension.
306
307
308 Generating the Makefile and running make created a
309 directory called blib (which stands for ``build library'')
310 in the current working directory. This directory will
311 contain the shared library that we will build. Once we have
312 tested it, we can install it into its final
313 location.
314
315
316 Invoking the test script via make test
317 -I arguments so that it could find the various
318 files that are part of the extension. It is ''very''
319 important that while you are still testing extensions that
320 you use ''make test``. If you try to run the
321 test script all by itself, you will get a fatal error.
322 Another reason it is important to use ''make test``
323 to run your test script is that if you are testing an
324 upgrade to an already-existing version, using ''make
325 test
326
327
328 When Perl sees a use extension;, it searches for a
329 file with the same name as the use'd extension that
330 has a .pm suffix. If that file cannot be found, Perl dies
331 with a fatal error. The default search path is contained in
332 the @INC array.
333
334
335 In our case, Mytest.pm tells perl that it will need the
336 Exporter and Dynamic Loader extensions. It then sets the
337 @ISA and @EXPORT arrays and the
338 $VERSION scalar; finally it tells perl to bootstrap
339 the module. Perl will call its dynamic loader routine (if
340 there is one) and load the shared library.
341
342
343 The two arrays @ISA and @EXPORT are very
344 important. The @ISA array contains a list of other
345 packages in which to search for methods (or subroutines)
346 that do not exist in the current package. This is usually
347 only important for object-oriented extensions (which we will
348 talk about much later), and so usually doesn't need to be
349 modified.
350
351
352 The @EXPORT array tells Perl which of the
353 extension's variables and subroutines should be placed into
354 the calling package's namespace. Because you don't know if
355 the user has already used your variable and subroutine
356 names, it's vitally important to carefully select what to
357 export. Do ''not'' export method or variable names ''by
358 default'' without a good reason.
359
360
361 As a general rule, if the module is trying to be
362 object-oriented then don't export anything. If it's just a
363 collection of functions and variables, then you can export
364 them via another array, called @EXPORT_OK. This
365 array does not automatically place its subroutine and
366 variable names into the namespace unless the user
367 specifically requests that this be done.
368
369
370 See perlmod for more information.
371
372
373 The $VERSION variable is used to ensure that the
374 .pm file and the shared library are ``in sync'' with each
375 other. Any time you make changes to the .pm or .xs files,
376 you should increment the value of this
377 variable.
378
379
380 __Writing good test scripts__
381
382
383 The importance of writing good test scripts cannot be
384 overemphasized. You should closely follow the ``ok/not ok''
385 style that Perl itself uses, so that it is very easy and
386 unambiguous to determine the outcome of each test case. When
387 you find and fix a bug, make sure you add a test case for
388 it.
389
390
391 By running make test``, you ensure that your
392 test.pl script runs and uses the correct version of your
393 extension. If you have many test cases, you might want to
394 copy Perl's test style. Create a directory named ''t`` in
395 the extension's directory and append the suffix ''.t`` to
396 the names of your test files. When you run ''make
397 test
398
399
400 __EXAMPLE 3__
401
402
403 Our third extension will take one argument as its input,
404 round off that value, and set the ''argument'' to the
405 rounded value.
406
407
408 Add the following to the end of Mytest.xs:
409
410
411 void
412 round(arg)
413 double arg
414 CODE:
415 if (arg
416 Edit the Makefile.PL file so that the corresponding line looks like this:
417
418
419 'LIBS' =
420 Generate the Makefile and run make. Change the BEGIN block to print ``1..9'' and add the following to test.pl:
421
422
423 $i = -1.5;
424 Running make test
425
426
427 Notice that in these new test cases, the argument passed to
428 round was a scalar variable. You might be wondering if you
429 can round a constant or literal. To see what happens,
430 temporarily add the following line to test.pl:
431
432
433
434 Run make test
435
436
437 __What's new here?__
438
439
440 We've made some changes to Makefile.PL. In this case, we've
441 specified an extra library to be linked into the extension's
442 shared library, the math library libm in this case. We'll
443 talk later about how to write XSUBs that can call every
444 routine in a library.
445
446
447 The value of the function is not being passed back as the
448 function's return value, but by changing the value of the
449 variable that was passed into the function. You might have
450 guessed that when you saw that the return value of round is
451 of type ``void''.
452
453
454 __Input and Output Parameters__
455
456
457 You specify the parameters that will be passed into the
458 XSUB on the line(s) after you declare the
459 function's return value and name. Each input parameter line
460 starts with optional white space, and may have an optional
461 terminating semicolon.
462
463
464 The list of output parameters occurs at the very end of the
465 function, just before after the OUTPUT:
466 directive. The use of RETVAL tells Perl that
467 you wish to send this value back as the return value of the
468 XSUB function. In Example 3, we wanted the
469 ``return value'' placed in the original variable which we
470 passed in, so we listed it (and not RETVAL )
471 in the OUTPUT: section.
472
473
474 __The XSUBPP Program__
475
476
477 The __xsubpp__ program takes the XS code
478 in the .xs file and translates it into C code, placing it in
479 a file whose suffix is .c. The C code created makes heavy
480 use of the C functions within Perl.
481
482
483 __The TYPEMAP file__
484
485
486 The __xsubpp__ program uses rules to convert from Perl's
487 data types (scalar, array, etc.) to C's data types (int,
488 char, etc.). These rules are stored in the typemap file
2 perry 489 ($PERLLIB/!ExtUtils/typemap). This file is split into three
1 perry 490 parts.
491
492
493 The first section maps various C data types to a name, which
494 corresponds somewhat with the various Perl types. The second
495 section contains C code which __xsubpp__ uses to handle
496 input parameters. The third section contains C code which
497 __xsubpp__ uses to handle output parameters.
498
499
500 Let's take a look at a portion of the .c file created for
501 our extension. The file name is Mytest.c:
502
503
504 XS(XS_Mytest_round)
505 {
506 dXSARGS;
507 if (items != 1)
508 croak(
509 Notice the two lines commented with `` XXXXX ''. If you check the first section of the typemap file, you'll see that doubles are of type T_DOUBLE. In the INPUT section, an argument that is T_DOUBLE is assigned to the variable arg by calling the routine SvNV on something, then casting it to double, then assigned to the variable arg. Similarly, in the OUTPUT section, once arg has its final value, it is passed to the sv_setnv function to be passed back to the calling subroutine. These two functions are explained in perlguts; we'll talk more later about what that `` ''ST'' (0)'' means in the section on the argument stack.
510
511
512 __Warning about Output Arguments__
513
514
515 In general, it's not a good idea to write extensions that
516 modify their input parameters, as in Example 3. Instead, you
517 should probably return multiple values in an array and let
518 the caller handle them (we'll do this in a later example).
519 However, in order to better accommodate calling pre-existing
520 C routines, which often do modify their input parameters,
521 this behavior is tolerated.
522
523
524 __EXAMPLE 4__
525
526
527 In this example, we'll now begin to write XSUBs that will
528 interact with pre-defined C libraries. To begin with, we
529 will build a small library of our own, then let h2xs write
530 our .pm and .xs files for us.
531
532
533 Create a new directory called Mytest2 at the same level as
534 the directory Mytest. In the Mytest2 directory, create
535 another directory called mylib, and cd into that
536 directory.
537
538
539 Here we'll create some files that will generate a test
540 library. These will include a C source file and a header
541 file. We'll also create a Makefile.PL in this directory.
542 Then we'll make sure that running make at the Mytest2 level
543 will automatically run this Makefile.PL file and the
544 resulting Makefile.
545
546
547 In the mylib directory, create a file mylib.h that looks
548 like this:
549
550
551 #define TESTVAL 4
552 extern double foo(int, long, const char*);
553 Also create a file mylib.c that looks like this:
554
555
556 #include
557 double
558 foo(int a, long b, const char *c)
559 {
560 return (a + b + atof(c) + TESTVAL);
561 }
562 And finally create a file Makefile.PL that looks like this:
563
564
2 perry 565 use !ExtUtils::!MakeMaker;
1 perry 566 $Verbose = 1;
2 perry 567 !WriteMakefile(
1 perry 568 NAME =
569 sub MY::top_targets {
570 '
571 all :: static
572 pure_all :: static
573 static :: libmylib$(LIB_EXT)
574 libmylib$(LIB_EXT): $(O_FILES)
575 $(AR) cr libmylib$(LIB_EXT) $(O_FILES)
576 $(RANLIB) libmylib$(LIB_EXT)
577 ';
578 }
579 Make sure you use a tab and not spaces on the lines beginning with ``$( AR )'' and ``$( RANLIB )''. Make will not function properly if you use spaces. It has also been reported that the ``cr'' argument to $( AR ) is unnecessary on Win32 systems.
580
581
582 We will now create the main top-level Mytest2 files. Change
583 to the directory above Mytest2 and run the following
584 command:
585
586
587 % h2xs -O -n Mytest2 ./Mytest2/mylib/mylib.h
588 This will print out a warning about overwriting Mytest2, but that's okay. Our files are stored in Mytest2/mylib, and will be untouched.
589
590
591 The normal Makefile.PL that h2xs generates doesn't know
592 about the mylib directory. We need to tell it that there is
593 a subdirectory and that we will be generating a library in
594 it. Let's add the argument MYEXTLIB to the
2 perry 595 !WriteMakefile call so that it looks like this:
1 perry 596
597
2 perry 598 !WriteMakefile(
1 perry 599 'NAME' =
600 and then at the end add a subroutine (which will override the pre-existing subroutine). Remember to use a tab character to indent the line beginning with ``cd''!
601
602
603 sub MY::postamble {
604 '
605 $(MYEXTLIB): mylib/Makefile
606 cd mylib
607 Let's also fix the MANIFEST file so that it accurately reflects the contents of our extension. The single line that says ``mylib'' should be replaced by the following three lines:
608
609
610 mylib/Makefile.PL
611 mylib/mylib.c
612 mylib/mylib.h
613 To keep our namespace nice and unpolluted, edit the .pm file and change the variable @EXPORT to @EXPORT_OK. Finally, in the .xs file, edit the #include line to read:
614
615
616 #include
617 And also add the following function definition to the end of the .xs file:
618
619
620 double
621 foo(a,b,c)
622 int a
623 long b
624 const char * c
625 OUTPUT:
626 RETVAL
627 Now we also need to create a typemap file because the default Perl doesn't currently support the const char * type. Create a file called typemap in the Mytest2 directory and place the following in it:
628
629
630 const char * T_PV
631 Now run perl on the top-level Makefile.PL. Notice that it also created a Makefile in the mylib directory. Run make and watch that it does cd into the mylib directory and run make in there as well.
632
633
634 Now edit the test.pl script and change the
635 BEGIN block to print ``1..4'', and add the
636 following lines to the end of the script:
637
638
639 print
640 (When dealing with floating-point comparisons, it is best to not check for equality, but rather that the difference between the expected and actual result is below a certain amount (called epsilon) which is 0.01 in this case)
641
642
643 Run make test
644
645
646 __What has happened here?__
647
648
649 Unlike previous examples, we've now run h2xs on a real
650 include file. This has caused some extra goodies to appear
651 in both the .pm and .xs files.
652
653
654 In the .xs file, there's now a #include directive with the
655 absolute path to the mylib.h header file. We changed this to
656 a relative path so that we could move the extension
657 directory if we wanted to.
658
659
660 There's now some new C code that's been added to the .xs
661 file. The purpose of the constant routine is to
662 make the values that are #define'd in the header file
663 accessible by the Perl script (by calling either
664 TESTVAL or ). There's
665 also some XS code to allow calls to the
666 constant routine.
667
668
669 The .pm file originally exported the name TESTVAL
670 in the @EXPORT array. This could lead to name
671 clashes. A good rule of thumb is that if the #define is only
672 going to be used by the C routines themselves, and not by
673 the user, they should be removed from the @EXPORT
674 array. Alternately, if you don't mind using the ``fully
675 qualified name'' of a variable, you could move most or all
676 of the items from the @EXPORT array into the
677 @EXPORT_OK array.
678
679
680 If our include file had contained #include directives, these
681 would not have been processed by h2xs. There is no good
682 solution to this right now.
683
684
685 We've also told Perl about the library that we built in the
686 mylib subdirectory. That required only the addition of the
2 perry 687 MYEXTLIB variable to the !WriteMakefile call and the
1 perry 688 replacement of the postamble subroutine to cd into the
689 subdirectory and run make. The Makefile.PL for the library
690 is a bit more complicated, but not excessively so. Again we
691 replaced the postamble subroutine to insert our own code.
692 This code simply specified that the library to be created
693 here was a static archive library (as opposed to a
694 dynamically loadable library) and provided the commands to
695 build it.
696
697
698 __Anatomy of .xs file__
699
700
701 The .xs file of `` EXAMPLE 4'' contained some
702 new elements. To understand the meaning of these elements,
703 pay attention to the line which reads
704
705
706 MODULE = Mytest2 PACKAGE = Mytest2
707 Anything before this line is plain C code which describes which headers to include, and defines some convenience functions. No translations are performed on this part, apart from having embedded POD documentation skipped over (see perlpod) it goes into the generated output C file as is.
708
709
710 Anything after this line is the description of
711 XSUB functions. These descriptions are
712 translated by __xsubpp__ into C code which implements
713 these functions using Perl calling conventions, and which
714 makes these functions visible from Perl
715 interpreter.
716
717
718 Pay a special attention to the function constant.
719 This name appears twice in the generated .xs file: once in
720 the first part, as a static C function, the another time in
721 the second part, when an XSUB interface to
722 this static C function is defined.
723
724
725 This is quite typical for .xs files: usually the .xs file
726 provides an interface to an existing C function. Then this C
727 function is defined somewhere (either in an external
728 library, or in the first part of .xs file), and a Perl
729 interface to this function (i.e. ``Perl glue'') is described
730 in the second part of .xs file. The situation in ``
731 EXAMPLE 1'', `` EXAMPLE 2'',
732 and `` EXAMPLE 3'', when all the work is done
733 inside the ``Perl glue'', is somewhat of an exception rather
734 than the rule.
735
736
737 __Getting the fat out of XSUBs__
738
739
740 In `` EXAMPLE 4'' the second part of .xs file
741 contained the following description of an
742 XSUB:
743
744
745 double
746 foo(a,b,c)
747 int a
748 long b
749 const char * c
750 OUTPUT:
751 RETVAL
752 Note that in contrast with `` EXAMPLE 1'', `` EXAMPLE 2'' and `` EXAMPLE 3'', this description does not contain the actual ''code'' for what is done is done during a call to Perl function ''foo()''. To understand what is going on here, one can add a CODE section to this XSUB:
753
754
755 double
756 foo(a,b,c)
757 int a
758 long b
759 const char * c
760 CODE:
761 RETVAL = foo(a,b,c);
762 OUTPUT:
763 RETVAL
764 However, these two XSUBs provide almost identical generated C code: __xsubpp__ compiler is smart enough to figure out the CODE: section from the first two lines of the description of XSUB . What about OUTPUT: section? In fact, that is absolutely the same! The OUTPUT: section can be removed as well, ''as far as CODE: section or PPCODE: section'' is not specified: __xsubpp__ can see that it needs to generate a function call section, and will autogenerate the OUTPUT section too. Thus one can shortcut the XSUB to become:
765
766
767 double
768 foo(a,b,c)
769 int a
770 long b
771 const char * c
772 Can we do the same with an XSUB
773
774
775 int
776 is_even(input)
777 int input
778 CODE:
779 RETVAL = (input % 2 == 0);
780 OUTPUT:
781 RETVAL
782 of `` EXAMPLE 2''? To do this, one needs to define a C function int is_even(int input). As we saw in ``Anatomy of .xs file'', a proper place for this definition is in the first part of .xs file. In fact a C function
783
784
785 int
786 is_even(int arg)
787 {
788 return (arg % 2 == 0);
789 }
790 is probably overkill for this. Something as simple as a #define will do too:
791
792
793 #define is_even(arg) ((arg) % 2 == 0)
794 After having this in the first part of .xs file, the ``Perl glue'' part becomes as simple as
795
796
797 int
798 is_even(input)
799 int input
800 This technique of separation of the glue part from the workhorse part has obvious tradeoffs: if you want to change a Perl interface, you need to change two places in your code. However, it removes a lot of clutter, and makes the workhorse part independent from idiosyncrasies of Perl calling convention. (In fact, there is nothing Perl-specific in the above description, a different version of __xsubpp__ might have translated this to TCL glue or Python glue as well.)
801
802
803 __More about XSUB arguments__
804
805
806 With the completion of Example 4, we now have an easy way to
807 simulate some real-life libraries whose interfaces may not
808 be the cleanest in the world. We shall now continue with a
809 discussion of the arguments passed to the __xsubpp__
810 compiler.
811
812
813 When you specify arguments to routines in the .xs file, you
814 are really passing three pieces of information for each
815 argument listed. The first piece is the order of that
816 argument relative to the others (first, second, etc). The
817 second is the type of argument, and consists of the type
818 declaration of the argument (e.g., int, char*, etc). The
819 third piece is the calling convention for the argument in
820 the call to the library function.
821
822
823 While Perl passes arguments to functions by reference, C
824 passes arguments by value; to implement a C function which
825 modifies data of one of the ``arguments'', the actual
826 argument of this C function would be a pointer to the data.
827 Thus two C functions with declarations
828
829
830 int string_length(char *s);
831 int upper_case_char(char *cp);
832 may have completely different semantics: the first one may inspect an array of chars pointed by s, and the second one may immediately dereference cp and manipulate *cp only (using the return value as, say, a success indicator). From Perl one would use these functions in a completely different manner.
833
834
835 One conveys this info to __xsubpp__ by replacing
836 * before the argument by .
837 means that the argument should be passed to a
838 library function by its address. The above two function may
839 be XSUB-ified as
840
841
842 int
843 string_length(s)
844 char * s
845 int
846 upper_case_char(cp)
847 char
848 For example, consider:
849
850
851 int
852 foo(a,b)
853 char
854 The first Perl argument to this function would be treated as a char and assigned to the variable a, and its address would be passed into the function foo. The second Perl argument would be treated as a string pointer and assigned to the variable b. The ''value'' of b would be passed into the function foo. The actual call to the function foo that __xsubpp__ generates would look like this:
855
856
857 foo(
858 __xsubpp__ will parse the following function argument lists identically:
859
860
861 char
862 However, to help ease understanding, it is suggested that you place a ``
863
864
865 You should take great pains to try to pass the function the
866 type of variable it wants, when possible. It will save you a
867 lot of trouble in the long run.
868
869
870 __The Argument Stack__
871
872
873 If we look at any of the C code generated by any of the
874 examples except example 1, you will notice a number of
875 references to ST (n), where n is usually 0.
876 `` ST '' is actually a macro that points to
877 the n'th argument on the argument stack.
878 ''ST'' (0) is thus the first argument on
879 the stack and therefore the first argument passed to the
880 XSUB , ''ST'' (1) is the
881 second argument, and so on.
882
883
884 When you list the arguments to the XSUB in
885 the .xs file, that tells __xsubpp__ which argument
886 corresponds to which of the argument stack (i.e., the first
887 one listed is the first argument, and so on). You invite
888 disaster if you do not list them in the same order as the
889 function expects them.
890
891
892 The actual values on the argument stack are pointers to the
893 values passed in. When an argument is listed as being an
894 OUTPUT value, its corresponding value on the
895 stack (i.e., ''ST'' (0) if it was the
896 first argument) is changed. You can verify this by looking
897 at the C code generated for Example 3. The code for the
898 ''round()'' XSUB routine contains lines
899 that look like this:
900
901
902 double arg = (double)SvNV(ST(0));
903 /* Round the contents of the variable arg */
904 sv_setnv(ST(0), (double)arg);
905 The arg variable is initially set by taking the value from ''ST'' (0), then is stored back into ''ST'' (0) at the end of the routine.
906
907
908 XSUBs are also allowed to return lists, not just scalars.
909 This must be done by manipulating stack values
910 ''ST'' (0), ''ST'' (1),
911 etc, in a subtly different way. See perlxs for
912 details.
913
914
915 XSUBs are also allowed to avoid automatic conversion of Perl
916 function arguments to C function arguments. See perlxs for
917 details. Some people prefer manual conversion by inspecting
918 ST(i) even in the cases when automatic conversion
919 will do, arguing that this makes the logic of an
920 XSUB call clearer. Compare with ``Getting the
921 fat out of XSUBs'' for a similar tradeoff of a complete
922 separation of ``Perl glue'' and ``workhorse'' parts of an
923 XSUB .
924
925
926 While experts may argue about these idioms, a novice to Perl
927 guts may prefer a way which is as little Perl-guts-specific
928 as possible, meaning automatic conversion and automatic call
929 generation, as in ``Getting the fat out of XSUBs''. This
930 approach has the additional benefit of protecting the
931 XSUB writer from future changes to the Perl
932 API .
933
934
935 __Extending your Extension__
936
937
938 Sometimes you might want to provide some extra methods or
939 subroutines to assist in making the interface between Perl
940 and your extension simpler or easier to understand. These
941 routines should live in the .pm file. Whether they are
942 automatically loaded when the extension itself is loaded or
943 only loaded when called depends on where in the .pm file the
944 subroutine definition is placed. You can also consult
2 perry 945 !AutoLoader for an alternate way to store and load your extra
1 perry 946 subroutines.
947
948
949 __Documenting your Extension__
950
951
952 There is absolutely no excuse for not documenting your
953 extension. Documentation belongs in the .pm file. This file
954 will be fed to pod2man, and the embedded documentation will
955 be converted to the man page format, then placed in the blib
956 directory. It will be copied to Perl's man page directory
957 when the extension is installed.
958
959
960 You may intersperse documentation and Perl code within the
961 .pm file. In fact, if you want to use method autoloading,
962 you must do this, as the comment inside the .pm file
963 explains.
964
965
966 See perlpod for more information about the pod
967 format.
968
969
970 __Installing your Extension__
971
972
973 Once your extension is complete and passes all its tests,
974 installing it is quite simple: you simply run ``make
975 install''. You will either need to have write permission
976 into the directories where Perl is installed, or ask your
977 system administrator to run the make for you.
978
979
980 Alternately, you can specify the exact directory to place
981 the extension's files by placing a
982 ``PREFIX=/destination/directory'' after the make install.
983 (or in between the make and install if you have a brain-dead
984 version of make). This can be very useful if you are
985 building an extension that will eventually be distributed to
986 multiple systems. You can then just archive the files in the
987 destination directory and distribute them to your
988 destination systems.
989
990
991 __EXAMPLE 5__
992
993
994 In this example, we'll do some more work with the argument
995 stack. The previous examples have all returned only a single
996 value. We'll now create an extension that returns an
997 array.
998
999
1000 This extension is very Unix-oriented (struct statfs and the
1001 statfs system call). If you are not running on a Unix
1002 system, you can substitute for statfs any other function
1003 that returns multiple values, you can hard-code values to be
1004 returned to the caller (although this will be a bit harder
1005 to test the error case), or you can simply not do this
1006 example. If you change the XSUB , be sure to
1007 fix the test cases to match the changes.
1008
1009
1010 Return to the Mytest directory and add the following code to
1011 the end of Mytest.xs:
1012
1013
1014 void
1015 statfs(path)
1016 char * path
1017 INIT:
1018 int i;
1019 struct statfs buf;
1020 PPCODE:
1021 i = statfs(path,
1022 You'll also need to add the following code to the top of the .xs file, just after the include of `` XSUB .h'':
1023
1024
1025 #include
1026 Also add the following code segment to test.pl while incrementing the ``1..9'' string in the BEGIN block to ``1..11'':
1027
1028
1029 @a =
1030
1031
1032 __New Things in this Example__
1033
1034
1035 This example added quite a few new concepts. We'll take them
1036 one at a time.
1037
1038
1039 The INIT: directive contains code that will
1040 be placed immediately after the argument stack is decoded. C
1041 does not allow variable declarations at arbitrary locations
1042 inside a function, so this is usually the best way to
1043 declare local variables needed by the XSUB .
1044 (Alternatively, one could put the whole PPCODE:
1045 section into braces, and put these declarations on
1046 top.)
1047
1048
1049 This routine also returns a different number of arguments
1050 depending on the success or failure of the call to statfs.
1051 If there is an error, the error number is returned as a
1052 single-element array. If the call is successful, then a
1053 9-element array is returned. Since only one argument is
1054 passed into this function, we need room on the stack to hold
1055 the 9 values which may be returned.
1056
1057
1058 We do this by using the PPCODE: directive,
1059 rather than the CODE: directive. This tells
1060 __xsubpp__ that we will be managing the return values
1061 that will be put on the argument stack by
1062 ourselves.
1063
1064
1065 When we want to place values to be returned to the caller
1066 onto the stack, we use the series of macros that begin with
1067 `` XPUSH ''. There are five different
1068 versions, for placing integers, unsigned integers, doubles,
1069 strings, and Perl scalars on the stack. In our example, we
1070 placed a Perl scalar onto the stack. (In fact this is the
1071 only macro which can be used to return multiple
1072 values.)
1073
1074
1075 The XPUSH* macros will automatically extend the return stack
1076 to prevent it from being overrun. You push values onto the
1077 stack in the order you want them seen by the calling
1078 program.
1079
1080
1081 The values pushed onto the return stack of the
1082 XSUB are actually mortal SV
1083 's. They are made mortal so that once the values are copied
1084 by the calling program, the SV 's that held
1085 the returned values can be deallocated. If they were not
1086 mortal, then they would continue to exist after the
1087 XSUB routine returned, but would not be
1088 accessible. This is a memory leak.
1089
1090
1091 If we were interested in performance, not in code
1092 compactness, in the success branch we would not use
1093 XPUSHs macros, but PUSHs macros, and would
1094 pre-extend the stack before pushing the return
1095 values:
1096
1097
1098 EXTEND(SP, 9);
1099 The tradeoff is that one needs to calculate the number of return values in advance (though overextending the stack will not typically hurt anything but memory consumption).
1100
1101
1102 Similarly, in the failure branch we could use PUSHs
1103 ''without'' extending the stack: the Perl function
1104 reference comes to an XSUB on the stack, thus
1105 the stack is ''always'' large enough to take one return
1106 value.
1107
1108
1109 __EXAMPLE 6__
1110
1111
1112 In this example, we will accept a reference to an array as
1113 an input parameter, and return a reference to an array of
1114 hashes. This will demonstrate manipulation of complex Perl
1115 data types from an XSUB .
1116
1117
1118 This extension is somewhat contrived. It is based on the
1119 code in the previous example. It calls the statfs function
1120 multiple times, accepting a reference to an array of
1121 filenames as input, and returning a reference to an array of
1122 hashes containing the data for each of the
1123 filesystems.
1124
1125
1126 Return to the Mytest directory and add the following code to
1127 the end of Mytest.xs:
1128
1129
1130 SV *
1131 multi_statfs(paths)
1132 SV * paths
1133 INIT:
1134 AV * results;
1135 I32 numpaths = 0;
1136 int i, n;
1137 struct statfs buf;
1138 if ((!SvROK(paths))
1139 (SvTYPE(SvRV(paths)) != SVt_PVAV)
1140 ((numpaths = av_len((AV *)SvRV(paths)))
1141 i = statfs(fn,
1142 rh = (HV *)sv_2mortal((SV *)newHV());
1143 hv_store(rh,
1144 av_push(results, newRV((SV *)rh));
1145 }
1146 RETVAL = newRV((SV *)results);
1147 OUTPUT:
1148 RETVAL
1149 And add the following code to test.pl, while incrementing the ``1..11'' string in the BEGIN block to ``1..13'':
1150
1151
1152 $results = Mytest::multi_statfs([[ '/', '/blech' ]);
1153 print ((ref $results-
1154
1155
1156 __New Things in this Example__
1157
1158
1159 There are a number of new concepts introduced here,
1160 described below:
1161
1162
1163 This function does not use a typemap. Instead, we declare it
1164 as accepting one SV* (scalar) parameter, and returning an
1165 SV* value, and we take care of populating these scalars
1166 within the code. Because we are only returning one value, we
1167 don't need a PPCODE: directive - instead, we use
1168 CODE: and OUTPUT: directives.
1169
1170
1171 When dealing with references, it is important to handle them
1172 with caution. The INIT: block first checks that
1173 SvROK returns true, which indicates that paths is a
1174 valid reference. It then verifies that the object referenced
1175 by paths is an array, using SvRV to dereference
1176 paths, and SvTYPE to discover its type. As an added
1177 test, it checks that the array referenced by paths is
1178 non-empty, using the av_len function (which returns
1179 -1 if the array is empty). The XSRETURN_UNDEF
1180 macro is used to abort the XSUB and return
1181 the undefined value whenever all three of these conditions
1182 are not met.
1183
1184
1185 We manipulate several arrays in this XSUB .
1186 Note that an array is represented internally by an AV*
1187 pointer. The functions and macros for manipulating arrays
1188 are similar to the functions in Perl: av_len
1189 returns the highest index in an AV*, much like $#array;
1190 av_fetch fetches a single scalar value from an
1191 array, given its index; av_push pushes a scalar
1192 value onto the end of the array, automatically extending the
1193 array as necessary.
1194
1195
1196 Specifically, we read pathnames one at a time from the input
1197 array, and store the results in an output array (results) in
1198 the same order. If statfs fails, the element pushed onto the
1199 return array is the value of errno after the failure. If
1200 statfs succeeds, though, the value pushed onto the return
1201 array is a reference to a hash containing some of the
1202 information in the statfs structure.
1203
1204
1205 As with the return stack, it would be possible (and a small
1206 performance win) to pre-extend the return array before
1207 pushing data into it, since we know how many elements we
1208 will return:
1209
1210
1211 av_extend(results, numpaths);
1212
1213
1214 We are performing only one hash operation in this function,
1215 which is storing a new scalar under a key using
1216 hv_store. A hash is represented by an HV* pointer.
1217 Like arrays, the functions for manipulating hashes from an
1218 XSUB mirror the functionality available from
1219 Perl. See perlguts and perlapi for details.
1220
1221
1222 To create a reference, we use the newRV function.
1223 Note that you can cast an AV* or an HV* to type SV* in this
1224 case (and many others). This allows you to take references
1225 to arrays, hashes and scalars with the same function.
1226 Conversely, the SvRV function always returns an
1227 SV*, which may need to be be cast to the appropriate type if
1228 it is something other than a scalar (check with
1229 SvTYPE).
1230
1231
1232 At this point, xsubpp is doing very little work - the
1233 differences between Mytest.xs and Mytest.c are
1234 minimal.
1235
1236
1237 __EXAMPLE 7 (Coming Soon)__
1238
1239
1240 XPUSH args AND set
1241 RETVAL AND assign return value to
1242 array
1243
1244
1245 __EXAMPLE 8 (Coming Soon)__
1246
1247
1248 Setting $!
1249
1250
1251 __EXAMPLE 9 (Coming Soon)__
1252
1253
1254 Getting fd's from filehandles
1255
1256
1257 __Troubleshooting these Examples__
1258
1259
1260 As mentioned at the top of this document, if you are having
1261 problems with these example extensions, you might see if any
1262 of these help you.
1263
1264
1265 In versions of 5.002 prior to the gamma version, the test
1266 script in Example 1 will not function properly. You need to
1267 change the ``use lib'' line to read:
1268
1269
1270 use lib './blib';
1271
1272
1273 In versions of 5.002 prior to version 5.002b1h, the test.pl
1274 file was not automatically created by h2xs. This means that
1275 you cannot say ``make test'' to run the test script. You
1276 will need to add the following line before the ``use
1277 extension'' statement:
1278
1279
1280 use lib './blib';
1281
1282
1283 In versions 5.000 and 5.001, instead of using the above
1284 line, you will need to use the following line:
1285
1286
1287 BEGIN { unshift(@INC,
1288
1289
1290 This document assumes that the executable named ``perl'' is
1291 Perl version 5. Some systems may have installed Perl version
1292 5 as ``perl5''.
1293 !!See also
1294
1295
1296 For more information, consult perlguts, perlapi, perlxs,
1297 perlmod, and perlpod.
1298 !!Author
1299
1300
1301 Jeff Okamoto okamoto@corp.hp.com''''
1302
1303
1304 Reviewed and assisted by Dean Roehrich, Ilya Zakharevich,
1305 Andreas Koenig, and Tim Bunce.
1306
1307
1308 __Last Changed__
1309
1310
1311 1999/11/30
1312 ----
This page is a man page (or other imported legacy content). We are unable to automatically determine the license status of this page.