+ if the export exits with error and as
+\begin_inset Quotes eld
+\end_inset
+
+failing
+\begin_inset Quotes erd
+\end_inset
+
+ if the export succeeds
+\emph on
+.
+
+\emph default
+ It follows that these expected failures will not show up as failed tests
+ in the test results and thus will not pollute the
+\begin_inset Quotes eld
+\end_inset
+
+good
+\begin_inset Quotes erd
+\end_inset
+
+ tests.
+ If the export actually succeeds, then the test will fail.
+ The purpose of this failure is to get your attention—something has changed,
+ possibly for the better.
+\end_layout
+
+\begin_layout Standard
+We try to document why a test is inverted or ignored.
+ See the comment (prefixed with
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+#
+\end_layout
+
+\end_inset
+
+) above the block in which the test is listed as inverted or ignored in
+ the files
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/suspiciousTests
+\end_layout
+
+\end_inset
+
+,
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/unreliableTests
+\end_layout
+
+\end_inset
+
+ and
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/ignoredTests
+\end_layout
+
+\end_inset
+
+.
+
+\end_layout
+
+\begin_layout Standard
+A good question is why do we enable the tests for non-default formats? The
+ answer is that if a non-default route is broken it is often because a bug
+ was introduced in LyX and not because a document-specific change was made
+ that is not supported by the route.
+ In other words, there is a high signal/noise ratio in the export tests
+ for some non-default formats.
+
+\end_layout
+
+\begin_layout Standard
+When a test or several tests fail, consider checking the files in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+Testing/Temporary/
+\end_layout
+
+\end_inset
+
+ subdirectory of your build directory.
+ In this subdirectory are three files: the file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+LastTestsFailed.log
+\end_layout
+
+\end_inset
+
+ simply lists the tests that failed on your last
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ command; the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+LastTest.log
+\end_layout
+
+\end_inset
+
+ file contains the output from the tests (and often has details explaining
+ why a test failed); and the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+CTestCostData.txt
+\end_layout
+
+\end_inset
+
+ file lists the times that it took to run the tests.
+\end_layout
+
+\begin_layout Paragraph
+What action should you take if a test fails?
+\end_layout
+
+\begin_layout Standard
+\paragraph_spacing single
+It is always good to check manually why something fails and if it passes
+ if the PDF output is good.
+\end_layout
+
+\begin_layout Itemize
+Generally, if a change breaks compilation for the target format (for the
+ manuals pdf2) without solving some important other issue,
+\emph on
+fix or revert the commit
+\emph default
+ that led to failure.
+\end_layout
+
+\begin_layout Itemize
+If it is not possible to (immediately) fix the failure but there are reasons
+ not to revert the commit (e.g.
+ it fixes another more important issue),
+\emph on
+invert
+\emph default
+ the failing test case (see
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:Inverted-tests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_layout Itemize
+If an
+\emph on
+inverted
+\emph default
+ test case fails because the export now works,
+\emph on
+uninvert
+\emph default
+ the test by removing the labeling pattern from
+\begin_inset Quotes eld
+\end_inset
+
+suspiciousTests
+\begin_inset Quotes erd
+\end_inset
+
+) (see
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:Inverted-tests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_layout Itemize
+If the export did not fail previously but led to wrong output (PDF, say),
+ it is in fact an improvement when the test now fails, label it as
+\begin_inset Quotes eld
+\end_inset
+
+unreliable:wrong:output
+\begin_inset Quotes erd
+\end_inset
+
+ (
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:Unreliable-tests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_layout Itemize
+In case of tests failing due to missing requirements (when only a subset
+ of TeXLive is installed or a test labeled