193 lines
5.6 KiB
Groff
193 lines
5.6 KiB
Groff
|
.\"
|
||
|
|
.\" Automated Testing Framework (atf)
|
||
|
|
.\"
|
||
|
|
.\" Copyright (c) 2007 The NetBSD Foundation, Inc.
|
||
|
|
.\" All rights reserved.
|
||
|
|
.\"
|
||
|
|
.\" Redistribution and use in source and binary forms, with or without
|
||
|
|
.\" modification, are permitted provided that the following conditions
|
||
|
|
.\" are met:
|
||
|
|
.\" 1. Redistributions of source code must retain the above copyright
|
||
|
|
.\" notice, this list of conditions and the following disclaimer.
|
||
|
|
.\" 2. Redistributions in binary form must reproduce the above copyright
|
||
|
|
.\" notice, this list of conditions and the following disclaimer in the
|
||
|
|
.\" documentation and/or other materials provided with the distribution.
|
||
|
|
.\"
|
||
|
|
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
|
||
|
|
.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
|
||
|
|
.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
||
|
|
.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
|
||
|
|
.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
|
||
|
|
.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
||
|
|
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
|
||
|
|
.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||
|
|
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
|
||
|
|
.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
|
||
|
|
.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
|
||
|
|
.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||
|
|
.\"
|
||
|
|
.Dd August 28, 2010
|
||
|
|
.Dt ATF 7
|
||
|
|
.Os
|
||
|
|
.Sh NAME
|
||
|
|
.Nm ATF
|
||
|
|
.Nd introduction to the Automated Testing Framework
|
||
|
|
.Sh DESCRIPTION
|
||
|
|
.Em IMPORTANT: If you are here because you want to know how to run the tests in
|
||
|
|
.Em Pa __TESTSDIR__ ,
|
||
|
|
.Em you most likely want to read the
|
||
|
|
.Em Xr tests 7
|
||
|
|
.Em manual page instead.
|
||
|
|
.Pp
|
||
|
|
The Automated Testing Framework
|
||
|
|
.Pf ( Nm )
|
||
|
|
is a collection of libraries and utilities designed to ease unattended
|
||
|
|
application testing in the hands of developers and end users of a specific
|
||
|
|
piece of software.
|
||
|
|
.Pp
|
||
|
|
As regards developers,
|
||
|
|
.Nm
|
||
|
|
provides the necessary means to easily create
|
||
|
|
test suites composed of multiple test programs, which in turn are a
|
||
|
|
collection of test cases.
|
||
|
|
It also attempts to simplify the debugging of problems when these test
|
||
|
|
cases detect an error by providing as much information as possible
|
||
|
|
about the failure.
|
||
|
|
.Pp
|
||
|
|
As regards users, it simplifies the process of running the test suites and,
|
||
|
|
in special, encourages end users to run them often: they do not need to
|
||
|
|
have source trees around nor any other development tools installed to be
|
||
|
|
able to certify that a given piece of software works on their machine as
|
||
|
|
advertised.
|
||
|
|
.Pp
|
||
|
|
If your operating systems distributes
|
||
|
|
.Nm ,
|
||
|
|
it is possible that it provides an introductory
|
||
|
|
.Xr tests 7
|
||
|
|
manual page.
|
||
|
|
You are encouraged to read it now.
|
||
|
|
.Ss License
|
||
|
|
.Nm
|
||
|
|
is distributed under the terms of the TNF License, a 2-clause BSD license.
|
||
|
|
For more details please see:
|
||
|
|
.Bd -literal -offset indent
|
||
|
|
.Pa __DOCDIR__/COPYING
|
||
|
|
.Ed
|
||
|
|
.Ss Components
|
||
|
|
.Nm
|
||
|
|
is a highly modular piece of software.
|
||
|
|
It provides a couple of libraries to ease the implementation of test
|
||
|
|
programs: one for the C and C++ languages and another one for shell
|
||
|
|
scripts.
|
||
|
|
It also includes multiple small utilities that follow the principle of
|
||
|
|
doing a single thing but doing it right.
|
||
|
|
This section outlines which these components are.
|
||
|
|
.Pp
|
||
|
|
Public utilities:
|
||
|
|
.Bl -tag -width atfXtestXprogramXXXXX
|
||
|
|
.It Xr atf-check 1
|
||
|
|
Executes a command and checks that its exit code, its standard output
|
||
|
|
and its standard error output match pre-specified expected values.
|
||
|
|
.It Xr atf-config 1
|
||
|
|
Queries static configuration information.
|
||
|
|
.It Xr atf-report 1
|
||
|
|
Converts the output of
|
||
|
|
.Nm atf-run
|
||
|
|
to user-friendly and/or machine-parseable reports.
|
||
|
|
.It Xr atf-run 1
|
||
|
|
Automates the execution of a series of test programs and collects their
|
||
|
|
results in a unified report.
|
||
|
|
.It Xr atf-sh 1
|
||
|
|
Shell interpreter for shell-based test programs.
|
||
|
|
.El
|
||
|
|
.Pp
|
||
|
|
Programming interfaces:
|
||
|
|
.Bl -tag -width atfXtestXprogramXXXXX
|
||
|
|
.It Xr atf-c-api 3
|
||
|
|
C programming interface for test programs.
|
||
|
|
.It Xr atf-c++-api 3
|
||
|
|
C++ programming interface for test programs.
|
||
|
|
.It Xr atf-sh-api 3
|
||
|
|
POSIX shell programming interface for test programs.
|
||
|
|
.El
|
||
|
|
.Pp
|
||
|
|
Other:
|
||
|
|
.Bl -tag -width atfXtestXprogramXXXXX
|
||
|
|
.It Xr atf-formats 5
|
||
|
|
Description of the machine-parseable data formats used by the tools.
|
||
|
|
.It Xr atf-test-case 4
|
||
|
|
Generic description of test cases, independent of the language they are
|
||
|
|
implemented in.
|
||
|
|
.It Xr atf-test-program 1
|
||
|
|
Common interface provided by the test programs written using the
|
||
|
|
.Nm
|
||
|
|
libraries.
|
||
|
|
.El
|
||
|
|
.Ss Recommended reading order
|
||
|
|
For end users wishing to run tests:
|
||
|
|
.Bl -enum -compact
|
||
|
|
.It
|
||
|
|
.Xr tests 7
|
||
|
|
(only if provided by your operating system).
|
||
|
|
.It
|
||
|
|
.Xr atf-test-program 1
|
||
|
|
.It
|
||
|
|
.Xr atf-run 1
|
||
|
|
.It
|
||
|
|
.Xr atf-report 1
|
||
|
|
.It
|
||
|
|
.Xr atf-config 1
|
||
|
|
.El
|
||
|
|
.Pp
|
||
|
|
For developers wanting to write their own tests:
|
||
|
|
.Bl -enum -compact
|
||
|
|
.It
|
||
|
|
Everything recommended to users.
|
||
|
|
.It
|
||
|
|
.Xr atf-test-case 4
|
||
|
|
.It
|
||
|
|
.Xr atf-c-api 3
|
||
|
|
.It
|
||
|
|
.Xr atf-c++-api 3
|
||
|
|
.It
|
||
|
|
.Xr atf-sh-api 3
|
||
|
|
.It
|
||
|
|
.Xr atf-sh 1
|
||
|
|
.It
|
||
|
|
.Xr atf-check 1
|
||
|
|
.El
|
||
|
|
.Pp
|
||
|
|
For those interested in
|
||
|
|
.Nm
|
||
|
|
internals:
|
||
|
|
.Bl -enum -compact
|
||
|
|
.It
|
||
|
|
Everything recommended to users.
|
||
|
|
.It
|
||
|
|
Everything recommended to developers.
|
||
|
|
.It
|
||
|
|
.Xr atf-formats 5
|
||
|
|
.El
|
||
|
|
.Sh SEE ALSO
|
||
|
|
.Xr tests 7
|
||
|
|
.Sh HISTORY
|
||
|
|
.Nm
|
||
|
|
started as a Google Summer of Code 2007 project mentored by The NetBSD
|
||
|
|
Foundation.
|
||
|
|
Its original goal was to provide a testing framework for The NetBSD
|
||
|
|
Operating System, but it grew as an independent project because the
|
||
|
|
framework itself did not need to be tied to a specific operating system.
|
||
|
|
.Pp
|
||
|
|
For more details on this subject, please see:
|
||
|
|
.Bd -literal -offset indent
|
||
|
|
.Pa __DOCDIR__/NEWS
|
||
|
|
.Pa __DOCDIR__/ROADMAP
|
||
|
|
.Ed
|
||
|
|
.Sh AUTHORS
|
||
|
|
For more details on the people that made
|
||
|
|
.Nm
|
||
|
|
possible, please see:
|
||
|
|
.Bd -literal -offset indent
|
||
|
|
.Pa __DOCDIR__/AUTHORS
|
||
|
|
.Ed
|