package TAP::Harness;
use strict;
use Carp;
use File::Spec;
use File::Path;
use IO::Handle;
use TAP::Base;
use vars qw($VERSION @ISA);
@ISA = qw(TAP::Base);
=head1 NAME
TAP::Harness - Run test scripts with statistics
=head1 VERSION
Version 3.23
=cut
$VERSION = '3.23';
$ENV{HARNESS_ACTIVE} = 1;
$ENV{HARNESS_VERSION} = $VERSION;
END {
# For VMS.
delete $ENV{HARNESS_ACTIVE};
delete $ENV{HARNESS_VERSION};
}
=head1 DESCRIPTION
This is a simple test harness which allows tests to be run and results
automatically aggregated and output to STDOUT.
=head1 SYNOPSIS
use TAP::Harness;
my $harness = TAP::Harness->new( \%args );
$harness->runtests(@tests);
=cut
my %VALIDATION_FOR;
my @FORMATTER_ARGS;
sub _error {
my $self = shift;
return $self->{error} unless @_;
$self->{error} = shift;
}
BEGIN {
@FORMATTER_ARGS = qw(
directives verbosity timer failures comments errors stdout color
show_count normalize
);
%VALIDATION_FOR = (
lib => sub {
my ( $self, $libs ) = @_;
$libs = [$libs] unless 'ARRAY' eq ref $libs;
return [ map {"-I$_"} @$libs ];
},
switches => sub { shift; shift },
exec => sub { shift; shift },
merge => sub { shift; shift },
aggregator_class => sub { shift; shift },
formatter_class => sub { shift; shift },
multiplexer_class => sub { shift; shift },
parser_class => sub { shift; shift },
scheduler_class => sub { shift; shift },
formatter => sub { shift; shift },
jobs => sub { shift; shift },
test_args => sub { shift; shift },
ignore_exit => sub { shift; shift },
rules => sub { shift; shift },
sources => sub { shift; shift },
version => sub { shift; shift },
trap => sub { shift; shift },
);
for my $method ( sort keys %VALIDATION_FOR ) {
no strict 'refs';
if ( $method eq 'lib' || $method eq 'switches' ) {
*{$method} = sub {
my $self = shift;
unless (@_) {
$self->{$method} ||= [];
return wantarray
? @{ $self->{$method} }
: $self->{$method};
}
$self->_croak("Too many arguments to method '$method'")
if @_ > 1;
my $args = shift;
$args = [$args] unless ref $args;
$self->{$method} = $args;
return $self;
};
}
else {
*{$method} = sub {
my $self = shift;
return $self->{$method} unless @_;
$self->{$method} = shift;
};
}
}
for my $method (@FORMATTER_ARGS) {
no strict 'refs';
*{$method} = sub {
my $self = shift;
return $self->formatter->$method(@_);
};
}
}
##############################################################################
=head1 METHODS
=head2 Class Methods
=head3 C
my %args = (
verbosity => 1,
lib => [ 'lib', 'blib/lib', 'blib/arch' ],
)
my $harness = TAP::Harness->new( \%args );
The constructor returns a new C object. It accepts an
optional hashref whose allowed keys are:
=over 4
=item * C
Set the verbosity level:
1 verbose Print individual test results to STDOUT.
0 normal
-1 quiet Suppress some test output (mostly failures
while tests are running).
-2 really quiet Suppress everything but the tests summary.
-3 silent Suppress everything.
=item * C
Append run time for each test to output. Uses L if
available.
=item * C
Show test failures (this is a no-op if C is selected).
=item * C
Show test comments (this is a no-op if C is selected).
=item * C
Update the running test count during testing.
=item * C
Set to a true value to normalize the TAP that is emitted in verbose modes.
=item * C
Accepts a scalar value or array ref of scalar values indicating which
paths to allowed libraries should be included if Perl tests are
executed. Naturally, this only makes sense in the context of tests
written in Perl.
=item * C
Accepts a scalar value or array ref of scalar values indicating which
switches should be included if Perl tests are executed. Naturally, this
only makes sense in the context of tests written in Perl.
=item * C
A reference to an C<@INC> style array of arguments to be passed to each
test program.
test_args => ['foo', 'bar'],
if you want to pass different arguments to each test then you should
pass a hash of arrays, keyed by the alias for each test:
test_args => {
my_test => ['foo', 'bar'],
other_test => ['baz'],
}
=item * C
Attempt to produce color output.
=item * C
Typically, Perl tests are run through this. However, anything which
spits out TAP is fine. You can use this argument to specify the name of
the program (and optional switches) to run your tests with:
exec => ['/usr/bin/ruby', '-w']
You can also pass a subroutine reference in order to determine and
return the proper program to run based on a given test script. The
subroutine reference should expect the TAP::Harness object itself as the
first argument, and the file name as the second argument. It should
return an array reference containing the command to be run and including
the test file name. It can also simply return C, in which case
TAP::Harness will fall back on executing the test script in Perl:
exec => sub {
my ( $harness, $test_file ) = @_;
# Let Perl tests run.
return undef if $test_file =~ /[.]t$/;
return [ qw( /usr/bin/ruby -w ), $test_file ]
if $test_file =~ /[.]rb$/;
}
If the subroutine returns a scalar with a newline or a filehandle, it
will be interpreted as raw TAP or as a TAP stream, respectively.
=item * C
If C is true the harness will create parsers that merge STDOUT
and STDERR together for any processes they start.
=item * C
I.
If set, C must be a hashref containing the names of the
Ls to load and/or configure. The values are a
hash of configuration that will be accessible to to the source handlers via
L.
For example:
sources => {
Perl => { exec => '/path/to/custom/perl' },
File => { extensions => [ '.tap', '.txt' ] },
MyCustom => { some => 'config' },
}
The C parameter affects how C. To configure a formatter,
you currently need to instantiate it outside of L and pass it in
with the C parameter to L. This I be addressed by adding
a I parameter to L in the future.
=head2 C
L version C<0.30> supports C.
To load C plugins, you'll need to use the C
parameter to C, typically from your C. For example:
Module::Build->new(
module_name => 'MyApp',
test_file_exts => [qw(.t .tap .txt)],
use_tap_harness => 1,
tap_harness_args => {
sources => {
MyCustom => {},
File => {
extensions => ['.tap', '.txt'],
},
},
formatter => 'TAP::Formatter::HTML',
},
build_requires => {
'Module::Build' => '0.30',
'TAP::Harness' => '3.18',
},
)->create_build_script;
See L
=head2 C
L does not support L out-of-the-box.
=head2 C
L supports C plugins, and has a plugin system of its
own. See L, L and L
for more details.
=head1 WRITING PLUGINS
If you can't configure C to do what you want, and you can't find
an existing plugin, consider writing one.
The two primary use cases supported by L for plugins are I
and I