#LyX 2.2 created this file. For more info see http://www.lyx.org/
-\lyxformat 503
+\lyxformat 504
\begin_document
\begin_header
+\save_transient_properties true
\origin /systemlyxdir/doc/
\textclass scrartcl
\options BCOR8mm,captions=tableheading
\font_sf_scale 100 100
\font_tt_scale 100 100
\graphics default
-\default_output_format default
-\output_sync 0
+\default_output_format pdf2
+\output_sync 1
\bibtex_command default
\index_command default
\paperfontsize 12
source code distribution.
This document is not translated, since the development language of \SpecialChar LyX
is
- english.
+ English.
If you want to use \SpecialChar LyX
you don't need to read this manual.
However, if you want to learn more about how \SpecialChar LyX
package Any new math package that is automatically loaded needs a file format
update.
The reason for this is that there is no true ERT inset for math formulas:
- Each command is parsed, and if a user happens to defne a local command
+ Each command is parsed, and if a user happens to define a local command
with the same name as a command that triggers an automatic load of a package,
he needs to be able to switch off the automatic loading of that package.
This switch is stored by the
\end_layout
\begin_layout Enumerate
+\begin_inset CommandInset label
+LatexCommand label
+name "enu:Describe_format"
+
+\end_inset
+
Describe the new format in
\begin_inset Flex Code
status collapsed
\end_layout
\begin_layout Enumerate
+\begin_inset CommandInset label
+LatexCommand label
+name "enu:Add-an-entry"
+
+\end_inset
+
Add an entry to both format lists (for conversion and reversion) in
\begin_inset Newline newline
\end_inset
\end_inset
.
- Then, revert the change to
+ Look at the resulting changes using the command
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-LFUNs.lyx
+git diff
\end_layout
\end_inset
- because that file is meant to be generated separately:
+.
+ If anything looks surprising, please investigate.
+ Keep in mind that the case of
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-git checkout lib/doc/LFUNs.lyx
-\end_layout
-
-\end_inset
-
-
-\begin_inset Note Note
-status collapsed
-
-\begin_layout Plain Layout
-TODO: this step should be done within updatedocs.py
+LFUNs.lyx
\end_layout
\end_inset
-.
- Look at the resulting changes using the command
+ is special, because it is first generated with
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-git diff
+gen_lfuns.py
\end_layout
\end_inset
-.
- If anything looks surprising, please investigate.
+ before being converted to the latest format.
Finally, commit using
\begin_inset Flex Code
status collapsed
.
\end_layout
+\begin_layout Standard
+Note that we do not update the local layout of our
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+.lyx
+\end_layout
+
+\end_inset
+
+ files because users would then not be able to export to older formats.
+ For example, if a 2.2.0 user exported a template to 2.1.x format and tried
+ to open the file in LyX 2.1.x, there would be an error because the file would
+ contain a local layout whose format is too new.
+ The root reason for this is that we do not support converting layouts to
+ older layout formats, as we do for the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+.lyx
+\end_layout
+
+\end_inset
+
+ file format.
+\end_layout
+
\begin_layout Subsection
Updating the file format number of bind/ui files
\end_layout
\begin_layout Standard
+A change to the functionality of existing LFUNs can require a conversion
+ of
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+.bind
+\end_layout
+
+\end_inset
+
+ and
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+.ui
+\end_layout
+
+\end_inset
+
+ files, and therefore an increment of the LFUN format, as well as a conversion
+ of Info insets in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+.lyx
+\end_layout
+
+\end_inset
+
+ files for manuals.
+ The latter cannot be done automatically and requires also a \SpecialChar LyX
+ format increase
+ (think of e.g.
+ someone who might have made a set of \SpecialChar LyX
+ teaching manuals for use in their
+ own group)
+\begin_inset Foot
+status open
+
+\begin_layout Plain Layout
+\begin_inset Flex URL
+status open
+
+\begin_layout Plain Layout
+
+http://www.lyx.org/trac/ticket/9794
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Enumerate
+Increment the LFUN file format number in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/LyXAction.h
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Enumerate
+Implement the LFUN conversion in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+lib/scripts/prefs2prefs_lfuns.py
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Enumerate
See step
\begin_inset CommandInset ref
LatexCommand ref
.
\end_layout
+\begin_layout Enumerate
+Update Info insets in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+.lyx
+\end_layout
+
+\end_inset
+
+ files.
+ To do so, increment the \SpecialChar LyX
+ format and proceed as in
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "subsec:update_lyx_files"
+
+\end_inset
+
+, steps
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "enu:Describe_format"
+
+\end_inset
+
+-
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "enu:updatefiles"
+
+\end_inset
+
+.
+ In the lyx2lyx implementation (
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "enu:Add-an-entry"
+
+\end_inset
+
+th step), implement a conversion similar to the one in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+prefs2prefs_lfuns.py
+\end_layout
+
+\end_inset
+
+ above, as well as a corresponding reversion; for this one can use
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+convert_info_insets
+\end_layout
+
+\end_inset
+
+ from
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+lib/lyx2lyx/lyx2lyx_tools.py
+\end_layout
+
+\end_inset
+
+.
+
+\end_layout
+
\begin_layout Subsection
Backporting new styles to the stable version
\end_layout
\end_layout
\begin_layout Subsection
-\SpecialChar LyX
- tests
+unit tests
\end_layout
\begin_layout Standard
-Some tests are located in the
-\begin_inset Flex Code
-status collapsed
-
-\begin_layout Plain Layout
-development/autotests
-\end_layout
-
-\end_inset
-
- subfolder of the \SpecialChar LyX
- source code distribution.
-\end_layout
-
-\begin_layout Subsubsection
-Running the tests
+There are attempts to set up a suite of unit tests for LyX.
\end_layout
\begin_layout Standard
-cmake is required to run the \SpecialChar LyX
- tests, running them is not implemented for
- autotools.
- The \SpecialChar LyX
- tests can be run by the commands
-\begin_inset Flex Code
-status collapsed
-
-\begin_layout Plain Layout
-ctest
-\end_layout
-
-\end_inset
-
- (all platforms) or
-\begin_inset Flex Code
-status collapsed
-
-\begin_layout Plain Layout
-make test
-\end_layout
-
-\end_inset
-
- (when using a make based build system and not MSVC) in the
-\begin_inset Flex Code
-status collapsed
-
-\begin_layout Plain Layout
-autotests
-\end_layout
-
-\end_inset
-
- subfolder of the build directory.
+TODO: describe what is done and what is still to do.
\end_layout
\begin_layout Subsection
\end_layout
\begin_layout Subsection
-Export tests (cmake only)
-\end_layout
-
-\begin_layout Standard
-The export tests are integration tests.
- They take longer to run and are more likely to break than the tex2lyx tests.
- Nevertheless, they have caught many regressions and without a better alternativ
-e it is important to keep them up-to-date and understand how they work.
+ctest automatic tests
\end_layout
\begin_layout Standard
-The export tests require the
+Some tests are located in the
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-ctest
+development/autotests/
\end_layout
\end_inset
- command that comes with the
-\begin_inset Flex Code
-status collapsed
+ subfolder of the \SpecialChar LyX
+ source code distribution.
+
+\begin_inset Foot
+status open
\begin_layout Plain Layout
-cmake
-\end_layout
+The README document in this folder only describes the
+\begin_inset Quotes eld
+\end_inset
+keytests
+\begin_inset Quotes erd
\end_inset
- build system
+ subset of autotests!
\end_layout
-\begin_layout Subsubsection
-Expectations of LyX developers
-\end_layout
+\end_inset
-\begin_layout Standard
-Because the export tests are integration tests and take a long time to run,
- LyX developers are rarely expected to run all of the tests.
- Here are some good practices to follow by developers:
+
+\end_layout
+
+\begin_layout Standard
+These tests can be run by the commands
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ in the
+\emph on
+ build directory
+\emph default
+ (all platforms) or (when using a make based build system and not MSVC)
+
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+make test
+\end_layout
+
+\end_inset
+
+ in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+autotests/
+\end_layout
+
+\end_inset
+
+ subfolder of the
+\emph on
+ build directory
+\emph default
+.
+ The test logs are written to the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+Testing/Temporary/
+\end_layout
+
+\end_inset
+
+ subfolder of the
+\emph on
+
+\emph default
+build directory.
+
+\end_layout
+
+\begin_layout Subsubsection
+Export tests
+\end_layout
+
+\begin_layout Standard
+The export tests are integration tests.
+ They take longer to run and are more likely to break than the tex2lyx tests.
+ Nevertheless, they have caught many regressions and without a better alternativ
+e it is important to keep them up-to-date and understand how they work.
+\end_layout
+
+\begin_layout Standard
+The export tests
+\begin_inset Quotes eld
+\end_inset
+
+reuse
+\begin_inset Quotes erd
+\end_inset
+
+ documentation, template, and example documents.
+ In addition, there are a number of dedicated sample documents in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+autotests/export/
+\end_layout
+
+\end_inset
+
+ subfolder of the \SpecialChar LyX
+ source code distribution.
+ All samples are (after copying and eventual processing by scripts) exported
+ to various output formats via the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+—export-to
+\end_layout
+
+\end_inset
+
+ command line option.
+ The tests checks for errors reported by LyX.
+ (However, error-free export is no guarantee for an error-free output document.)
+\end_layout
+
+\begin_layout Paragraph
+Expectations of LyX developers
+\end_layout
+
+\begin_layout Standard
+Because the export tests are integration tests and take a long time to run,
+ LyX developers are rarely expected to run all of the tests.
+ Here are some good practices to follow by developers:
\end_layout
\begin_layout Itemize
\begin_layout Itemize
When making non-trivial changes to a .lyx file, run the export tests correspondin
g to that .lyx file.
+
+\begin_inset Foot
+status collapsed
+
+\begin_layout Plain Layout
+This rule is due to revision.
+
+\end_layout
+
+\begin_layout Plain Layout
+There is an objection from the documentation maintainer that working on
+ the documentation must not be complicated by having to consider non-standard
+ exports.
+\end_layout
+
+\begin_layout Itemize
+successful compiling/testing an edited documentation file with pdflatex
+ suffices to ensure it can be commited, not tests with other exports are
+ required.
+\end_layout
+
+\begin_layout Plain Layout
+If sudden failures with other exports due to “half-tested” documentation
+ updates are a problem for the test maintainer, the test suite should use
+ copies that are
+\end_layout
+
+\begin_layout Itemize
+copied to a cache dir (autotests/samples/doc/, say) but not changed,
+\end_layout
+
+\begin_layout Itemize
+updated regularely (but on a time chosen by the test suite maintainer) from
+ the originals in lib/doc/
+\end_layout
+
+\begin_layout Plain Layout
+This way,
+\end_layout
+
+\begin_layout Itemize
+no test will fail due to ongoing work on documentation,
+\end_layout
+
+\begin_layout Itemize
+the documentation is still tested in full (with some delay),
+\end_layout
+
+\begin_layout Itemize
+failures with non-default export can be examined and handled accordingly
+ in one run with the cache update,
\end_layout
\begin_layout Itemize
-When making non-trivial changes to LyX's LaTeX export code (e.g.
+“interesting failures” (like the nested-language+polyglossia problem in
+ es/Customization can be separated and moved into dedicated test samples.
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Itemize
+When making non-trivial changes to LyX's \SpecialChar LaTeX
+ export code (e.g.
touching the encoding code or package handling code that you expect will
- change the exported LaTeX in some way), consider running all of the export
- tests before and after your change.
+ change the exported \SpecialChar LaTeX
+ in some way):
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+\paragraph_spacing single
+Consider running all of the export tests before and after your change.
If there are differences, please reconcile these (i.e.
fix the bug or fix the tests)
\emph on
before
\emph default
committing.
- Ask for help if you're not sure what to do or if you do not want to run
- the tests, post the patch on the list and others will run the tests.
+ Ask for help if you're not sure what to.
+\end_layout
+
+\begin_layout Standard
+If you do not want to run the tests,
+\end_layout
+
+\begin_layout Itemize
+post the patch on the list and others will run the tests and eventually
+ ask for fixes, or
+\end_layout
+
+\begin_layout Itemize
+commit, but be prepared to fix eventually arising problems or to revert
+ the commit if there is no easy fix.
\end_layout
+\end_deeper
\begin_layout Itemize
Understand how to interpret test failures.
If your commit is found to have broken a test, you should be able to interpret
.
\end_layout
-\begin_layout Subsubsection
-Configuring the tests
+\begin_layout Paragraph
+\begin_inset CommandInset label
+LatexCommand label
+name "par:export-test-output-formats"
+
+\end_inset
+
+Output formats
\end_layout
\begin_layout Standard
-To enable these tests, add the
-\begin_inset Flex Code
-status collapsed
+The following output formats are currently tested for each sample document
+ (see
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:Export-test-filtering"
+
+\end_inset
+
+ for exceptions):
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+LyX:
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+lyx16 LyX 1.6 file format (lyx2lyx)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+lyx21 LyX 2.1 file format (lyx2lyx)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+xhtml LyXHTML (native LyX HTML export)
+\end_layout
+
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+LyX
+\begin_inset space ~
+\end_inset
+
++
+\begin_inset space ~
+\end_inset
+
+LaTeX:
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+dvi DVI (8-bit latex)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+dvi3_texF DVI (LuaTeX with 8-bit TeX fonts)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+dvi3_systemF DVI (LuaTeX with Unicode fonts)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+pdf2 PDF (pdflatex)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+pdf4_texF PDF (XeTeX with 8-bit TeX fonts)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+pdf4_systemF PDF (XeTeX with Unicode fonts)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+pdf5_texF PDF (LuaTeX with 8-bit TeX fonts)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+pdf5_systemF PDF (LuaTeX with Unicode fonts)
+\end_layout
+
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+LyX
+\begin_inset space ~
+\end_inset
+
++
+\begin_inset space ~
+\end_inset
+
+LaTeX
+\begin_inset space ~
+\end_inset
+
++
+\begin_inset space ~
+\end_inset
+
+postprocessing:
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+pdf DVI -> PS (dvips) -> PDF (ps2pdf)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+pdf3 DVI -> PDF (dvipdfm)
+\end_layout
+
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+not
+\begin_inset space ~
+\end_inset
+
+tested: (or only if set as default output format in the document source)
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+latex LaTeX (plain)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+luatex LaTeX (LuaTeX)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+dviluatex LaTeX (dviluatex)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+pdflatex LaTeX (pdflatex)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+platex LaTeX (pLaTeX)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+xetex LaTeX (XeTeX)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+eps3 EPS (encapsulated Postscript) (cropped)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+ps DVI -> Postscript (dvips)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+odf
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+text (nor text2, ..., text4)
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+textparagraph
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+word
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+word2
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring pdf5msystemFM
+wordhtml
+\end_layout
+
+\end_deeper
+\begin_layout Paragraph
+\begin_inset CommandInset label
+LatexCommand label
+name "par:Configuring-ctests"
+
+\end_inset
+
+Configuring the tests
+\end_layout
+
+\begin_layout Standard
+To enable the export autotests, add the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-DLYX_ENABLE_EXPORT_TESTS=ON
+\end_layout
+
+\end_inset
+
+ flag.
+ For example:
+\end_layout
+
+\begin_layout Standard
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+cmake -DLYX_ENABLE_EXPORT_TESTS=ON /path/to/source
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\noindent
+This flag will increase the time for the cmake command by several seconds,
+ mainly because of the process of inverting tests (see Section
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "subsec:Interpreting-export-tests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_layout Paragraph
+\begin_inset CommandInset label
+LatexCommand label
+name "par:ctest-options"
+
+\end_inset
+
+Running the tests
+\end_layout
+
+\begin_layout Standard
+To run all tests, in the build directory simply run the command
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+.
+ A full, up-to-date TeXLive installation is recommended to run the tests.
+ Otherwise, some tests will fail.
+ Tests with additional requirements are labeled
+\begin_inset Quotes eld
+\end_inset
+
+unreliable:nonstandard
+\begin_inset Quotes erd
+\end_inset
+
+.
+
+\end_layout
+
+\begin_layout Standard
+To run only some of the tests, use command line options (see examples below):
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-R <pattern>
+\end_layout
+
+\end_inset
+
+ Run only the tests whose names match the given regular expression.
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-L <pattern>
+\end_layout
+
+\end_inset
+
+ Run only the tests whose labels match the given regular expression.
+ A test may have more that one label.
+
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-E <pattern>
+\end_layout
+
+\end_inset
+
+ Exclude the tests whose names match the given regular expression.
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-LE <pattern>
+\end_layout
+
+\end_inset
+
+ Exclude the tests whose labels match the given regular expression.
+ Cannot be combined with
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-L
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Standard
+The following options help to find good selection patterns:
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-N
+\end_layout
+
+\end_inset
+
+ List the tests that would be run but not actually run them.
+
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+Useful in conjunction with the -R, -L, -E and -LE options, e.g., if you want
+ to know how many tests there are or whether your
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+<pattern>
+\end_layout
+
+\end_inset
+
+ regular expression did what you expected.
+\end_layout
+
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+\SpecialChar nobreakdash
+\SpecialChar nobreakdash
+print-labels
+\end_layout
+
+\end_inset
+
+ print the list of all labels associated with the test set.
+ Can also be combined with -R, -L, -E, ...
+
+\end_layout
+
+\begin_layout Standard
+Other useful options are:
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-j <jobs>
+\end_layout
+
+\end_inset
+
+ Run the tests in parallel using the given number of jobs.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+We are still working on getting the tests to run in parallel.
+ However, when running the tests in parallel, sometimes tests fail that
+ pass when run sequentially.
+ A reasonable approach is to first run the tests in parallel and then run
+ the failed tests sequentially.
+\end_layout
+
+\begin_layout Standard
+For example, to run 8 jobs at a time:
+\end_layout
+
+\begin_layout Standard
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -j8
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest \SpecialChar nobreakdash
+\SpecialChar nobreakdash
+rerun-failed
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+When specifying a subset of the tests (e.g.
+ using
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+\SpecialChar nobreakdash
+R <pattern>
+\end_layout
+
+\end_inset
+
+), the same subset must be specified when using the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+\SpecialChar nobreakdash
+\SpecialChar nobreakdash
+rerun-failed
+\end_layout
+
+\end_inset
+
+ option because it is the test numbers that are used to index which tests
+ failed on the previous run.
+\end_layout
+
+\begin_layout Standard
+\noindent
+Note that some tests cannot be run in parallel.
+ These tests are marked in the code with the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+\noindent
+RUN_SERIAL ON
+\end_layout
+
+\end_inset
+
+ CMake property.
+\end_layout
+
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+\SpecialChar nobreakdash
+\SpecialChar nobreakdash
+timeout <seconds>
+\end_layout
+
+\end_inset
+
+ Set a global timeout on all tests that do not already have a timeout set
+ on them.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+There have been bugs in LyX and in \SpecialChar LaTeX
+ which cause compilation to hang, and
+ without a timeout a test might never stop (in one case there was even a
+ memory leak).
+ If a test times out, the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ command exits with error, but you can distinguish between a timed out test
+ and a failed test in the output reported at the end of the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ command.
+\end_layout
+
+\end_deeper
+\begin_layout Standard
+See the manual (
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+man ctest
+\end_layout
+
+\end_inset
+
+) the full list of command line options.
+\end_layout
+
+\begin_layout Paragraph
+Examples
+\end_layout
+
+\begin_layout Itemize
+run only the export tests:
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L export
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Itemize
+run inverted tests:
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L "inverted|suspended"
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Itemize
+list all export tests which match any of the labelling patterns:
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -N -R "
+\backslash
+..*_export/"
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Itemize
+exclude rarely used output formats and post-processing tests
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L export -E "_(texF|dvi3|pdf3?)"
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Paragraph
+\begin_inset CommandInset label
+LatexCommand label
+name "subsec:Interpreting-export-tests"
+
+\end_inset
+
+Interpreting the export test results
+\end_layout
+
+\begin_layout Standard
+A test can fail for several reasons, not all of them bad.
+\end_layout
+
+\begin_layout Enumerate
+A new or edited sample document may be incompatible with some output formats.
+\end_layout
+
+\begin_layout Enumerate
+A dependency is not met (e.g.
+ the \SpecialChar LaTeX
+ class file).
+ One hint that this is the case is that the corresponding
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+check_load
+\end_layout
+
+\end_inset
+
+ test will likely also fail.
+\end_layout
+
+\begin_layout Enumerate
+An inverted test fails to fail (i.e.
+ export that previously failed now works).
+\end_layout
+
+\begin_layout Enumerate
+An external dependency was updated (e.g.
+ \SpecialChar TeX
+ Live).
+\end_layout
+
+\begin_layout Enumerate
+A recent code change introduced a bug.
+\end_layout
+
+\begin_layout Enumerate
+\begin_inset CommandInset label
+LatexCommand label
+name "enu:exposed"
+
+\end_inset
+
+A change in a document exposed a previously unknown bug or an incompatibility
+ with an export format (e.g.
+ Lua\SpecialChar LaTeX
+).
+\end_layout
+
+\begin_layout Standard
+Because the .lyx files are exported in several formats, it is not surprising
+ that many of the exports fail.
+ This expectation of failure is addressed by
+\begin_inset Quotes eld
+\end_inset
+
+inverting
+\begin_inset Quotes erd
+\end_inset
+
+ the tests, that is, by marking the test as
+\begin_inset Quotes eld
+\end_inset
+
+passing
+\begin_inset Quotes erd
+\end_inset
+
+ if the export exits with error and as
+\begin_inset Quotes eld
+\end_inset
+
+failing
+\begin_inset Quotes erd
+\end_inset
+
+ if the export succeeds
+\emph on
+.
+
+\emph default
+ It follows that these expected failures will not show up as failed tests
+ in the test results and thus will not pollute the
+\begin_inset Quotes eld
+\end_inset
+
+good
+\begin_inset Quotes erd
+\end_inset
+
+ tests.
+ If the export actually succeeds, then the test will fail.
+ The purpose of this failure is to get your attention—something has changed,
+ possibly for the better.
+\end_layout
+
+\begin_layout Standard
+We try to document why a test is inverted or ignored.
+ See the comment (prefixed with
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+#
+\end_layout
+
+\end_inset
+
+) above the block in which the test is listed as inverted or ignored in
+ the files
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/suspiciousTests
+\end_layout
+
+\end_inset
+
+,
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/unreliableTests
+\end_layout
+
+\end_inset
+
+ and
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/ignoredTests
+\end_layout
+
+\end_inset
+
+.
+
+\end_layout
+
+\begin_layout Standard
+A good question is why do we enable the tests for non-default formats? The
+ answer is that if a non-default route is broken it is often because a bug
+ was introduced in LyX and not because a document-specific change was made
+ that is not supported by the route.
+ In other words, there is a high signal/noise ratio in the export tests
+ for some non-default formats.
+
+\end_layout
+
+\begin_layout Standard
+When a test or several tests fail, consider checking the files in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+Testing/Temporary/
+\end_layout
+
+\end_inset
+
+ subdirectory of your build directory.
+ In this subdirectory are three files: the file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+LastTestsFailed.log
+\end_layout
+
+\end_inset
+
+ simply lists the tests that failed on your last
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ command; the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+LastTest.log
+\end_layout
+
+\end_inset
+
+ file contains the output from the tests (and often has details explaining
+ why a test failed); and the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+CTestCostData.txt
+\end_layout
+
+\end_inset
+
+ file lists the times that it took to run the tests.
+\end_layout
+
+\begin_layout Paragraph
+What action should you take if a test fails?
+\end_layout
+
+\begin_layout Standard
+\paragraph_spacing single
+It is always good to check manually why something fails and if it passes
+ if the PDF output is good.
+\end_layout
+
+\begin_layout Itemize
+Generally, if a change breaks compilation for the target format (for the
+ manuals pdf2) without solving some important other issue,
+\emph on
+fix or revert the commit
+\emph default
+ that led to failure.
+\end_layout
+
+\begin_layout Itemize
+If it is not possible to (immediately) fix the failure but there are reasons
+ not to revert the commit (e.g.
+ it fixes another more important issue),
+\emph on
+invert
+\emph default
+ the failing test case (see
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:Inverted-tests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_layout Itemize
+If an
+\emph on
+inverted
+\emph default
+ test case fails because the export now works,
+\emph on
+uninvert
+\emph default
+ the test by removing the labeling pattern from
+\begin_inset Quotes eld
+\end_inset
+
+suspiciousTests
+\begin_inset Quotes erd
+\end_inset
+
+) (see
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:Inverted-tests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_layout Itemize
+If the export did not fail previously but led to wrong output (PDF, say),
+ it is in fact an improvement when the test now fails, label it as
+\begin_inset Quotes eld
+\end_inset
+
+unreliable:wrong:output
+\begin_inset Quotes erd
+\end_inset
+
+ (
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:Unreliable-tests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_layout Itemize
+In case of tests failing due to missing requirements (when only a subset
+ of TeXLive is installed or a test labeled
+\begin_inset Quotes eld
+\end_inset
+
+unreliable:nonstandard
+\begin_inset Quotes erd
+\end_inset
+
+ fails), ignore the failure, ask for someone else to run the test, or install
+ the missing ressources and try again.
+\end_layout
+
+\begin_layout Paragraph
+\begin_inset CommandInset label
+LatexCommand label
+name "par:Inverted-tests"
+
+\end_inset
+
+Inverted tests
+\end_layout
+
+\begin_layout Standard
+Test cases whose name matches a pattern in the file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/suspiciousTests
+\end_layout
+
+\end_inset
+
+ get the label
+\emph on
+inverted
+\emph default
+.
+ They get also the test property
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+WILL_FAIL
+\end_layout
+
+\end_inset
+
+, i.e.
+ they are reported as failing if the export works without error
+\begin_inset Flex URL
+status collapsed
+
+\begin_layout Plain Layout
+
+https://cmake.org/cmake/help/v3.0/command/set_tests_properties.html
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Standard
+Add failing cases to this file, if they cannot be solved
+\begin_inset Quotes eld
+\end_inset
+
+immediately
+\begin_inset Quotes erd
+\end_inset
+
+ but it is expected that the export will work in a foreseeable future, e.g.
+ low priority issues like failures to export to a non-target format (for
+ the manuals everything except pdf2).
+\end_layout
+
+\begin_layout Standard
+The following sublabels are currently present in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+suspiciousTests
+\end_layout
+
+\end_inset
+
+:
+\end_layout
+
+\begin_layout Description
+todo test failures that require attention:
+\end_layout
+
+\begin_deeper
+\begin_layout Itemize
+minor issues to explore and properly sort later,
+\end_layout
+
+\begin_layout Itemize
+easyfix issues,
+\end_layout
+
+\begin_layout Itemize
+LyX bugs to report at trac (move pattern to section "lyxbugs" once done).
+\end_layout
+
+\end_deeper
+\begin_layout Description
+lyxbugs LyX bugs with a Trac number.
+\end_layout
+
+\begin_layout Description
+ert Export failures due to "raw" LaTeX use in ERT or preamble code.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+"Wontfix" if demonstrating correct use and OK in the default output format.
+\end_layout
+
+\end_deeper
+\begin_layout Description
+texissues Export fails due to LaTeX limitations like non-ASCII characters
+ in verbatim or listings, incompatible packages, ...
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+"Wontfix" if documents demonstrate correct use in the default output format:
+\end_layout
+
+\begin_layout Itemize
+If the source can be made more robust without becoming "hackish", fix the
+ source,
+\end_layout
+
+\begin_layout Itemize
+if LyX could be enhanced to care for a permanent TeX limitation, file a
+ ticket at trac and add a pattern under lyxbugs,
+\end_layout
+
+\begin_layout Itemize
+otherwise, add a pattern here.
+\end_layout
+
+\end_deeper
+\begin_layout Description
+attic Documents in the attic.
+ (Kept for reference and format conversion test.)
+\end_layout
+
+\begin_layout Subparagraph
+suspended
+\end_layout
+
+\begin_layout Standard
+Test cases whose name additionally matches a pattern in the file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/suspendedTests
+\end_layout
+
+\end_inset
+
+ get the label
+\emph on
+suspended
+\emph default
+(instead of
+\emph on
+export
+\emph default
+and
+\emph on
+ inverted
+\emph default
+).
+ This means they are not executed using
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L export
+\end_layout
+
+\end_inset
+
+ or
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L inverted
+\end_layout
+
+\end_inset
+
+.
+ However, they also get the test property
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+WILL_FAIL
+\end_layout
+
+\end_inset
+
+, i.e.
+ they are reported as failing if the export works without error.
+ From time to time they still have to be checked using
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L suspended
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Standard
+These tests are suspended, because the export fails for known reasons which
+ cannot ATM be resolved.
+ But it is expected the reason might disappear in the future.
+ Be it new TL or better handling in \SpecialChar LyX
+.
+\end_layout
+
+\begin_layout Standard
+For ctest commands without the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-L
+\end_layout
+
+\end_inset
+
+ parameter nothing changes.
+ Suspended or not, tests will be executed depending only on the selecting
+ regular expression given to the ctest command (see
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:ctest-options"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_layout Paragraph
+\begin_inset CommandInset label
+LatexCommand label
+name "par:Unreliable-tests"
+
+\end_inset
+
+Unreliable tests
+\end_layout
+
+\begin_layout Standard
+Test cases whose name matches a pattern in the file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/unreliableTests
+\end_layout
+
+\end_inset
+
+ get the label
+\emph on
+unreliable
+\emph default
+.
+\end_layout
+
+\begin_layout Standard
+These tests are not executed using
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L export
+\end_layout
+
+\end_inset
+
+ or
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L inverted
+\end_layout
+
+\end_inset
+
+.
+
+\end_layout
+
+\begin_layout Standard
+They pass or fail for various reasons not related to LyX (nonstandard, erratic)
+ or pass but should rather fail (wrong output).
+
+\begin_inset Note Note
+status collapsed
+
+\begin_layout Plain Layout
+*invalid* tests (wrong output) are not *unreliable*.
+ # Use "unfit" or "unapplicable" as better label and name of pattern file?
+
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Description
+nonstandard Documents with additional requirements, e.g.
+ a class or package file not in TeXLive.
+
+\begin_inset Note Note
+status open
+
+\begin_layout Plain Layout
+TODO: rename to "extra"?
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+These tests are labeled as
+\family typewriter
+'nonstandard'.
+\end_layout
+
+\end_deeper
+\begin_layout Description
+erratic Tests depending on local configuration, OS, TeX distribution, package
+ versions, or the phase of the moon.
+
+\begin_inset Note Note
+status open
+
+\begin_layout Plain Layout
+TODO: use
+\emph on
+erratic
+\emph default
+ only for the phase-of-moon dependency?
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+These tests are labeled as
+\family typewriter
+'erratic'.
+\end_layout
+
+\end_deeper
+\begin_layout Description
+wrong
+\begin_inset space ~
+\end_inset
+
+output Export does not fail but the resulting document has errors.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+\paragraph_spacing single
+\begin_inset Note Note
+status open
+
+\begin_layout Plain Layout
+\paragraph_spacing single
+These tests are actually not
+\emph on
+unreliable
+\emph default
+ but
+\emph on
+invalid
+\emph default
+ (not measuring what they should measure).
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\end_deeper
+\begin_layout Paragraph
+\begin_inset CommandInset label
+LatexCommand label
+name "par:Export-test-filtering"
+
+\end_inset
+
+Export test filtering
+\end_layout
+
+\begin_layout Standard
+The assignment of a label to a test is controlled by a set of files with
+ regular expressions that are matched against the test names.
+\end_layout
+
+\begin_layout Description
+ignoredTests (small file)
+\begin_inset Newline newline
+\end_inset
+
+Tests selected here are withdrawn in the configuration step (cf.
+
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:Configuring-ctests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Input Test of any export combination
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Output Stop if tests not selected here
+\end_layout
+
+\end_deeper
+\begin_layout Description
+unreliableTests: Tests selected either pass or fail, but that is dependent
+ on the system where the test is run.
+ Selected tests gain the label 'unreliable'.
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Input Each test which passed 'ignoredTests'
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Output Stop if test selected, gain label 'unreliable'.
+\end_layout
+
+\end_deeper
+\begin_layout Description
+suspiciousTests
+\begin_inset space \space{}
+\end_inset
+
+
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Input Each test which passed 'unreliableTests'
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Output Stop if not selected.
+\end_layout
+
+\begin_layout Standard
+The following file is meant as subselections of 'suspiciousTests'.
+ If neither subselection applies, test gains labels 'export' and 'inverted'
+\end_layout
+
+\begin_layout Description
+suspendedTests Tests selected here gain the label 'suspended' but _not_
+ 'export' or 'inverted', although in ctest they remain inverted.
+ ('ctest' knows only 'inverted' or not, labels are used only for test selection)
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Input Each test selected by 'suspiciousTests'
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Output Selected test gains label 'suspended'.
+
+\end_layout
+
+\end_deeper
+\end_deeper
+\begin_layout Standard
+The following table may clarify label assignement
+\end_layout
+
+\begin_layout Standard
+\begin_inset Tabular
+<lyxtabular version="3" rows="7" columns="12">
+<features tabularvalignment="middle">
+<column alignment="left" valignment="top" width="0pt">
+<column alignment="left" valignment="top" width="0pt">
+<column alignment="left" valignment="top" width="0pt">
+<column alignment="left" valignment="top" width="0pt">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<row>
+<cell multicolumn="1" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+Test found in file:
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+Marked in ctest, Assigned label
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
-\begin_layout Plain Layout
--DLYX_ENABLE_EXPORT_TESTS=ON
\end_layout
\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
- flag.
- For example:
\end_layout
-\begin_layout Standard
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-cmake -DLYX_ENABLE_EXPORT_TESTS=ON /path/to/source
+
\end_layout
\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+\begin_layout Plain Layout
\end_layout
-\begin_layout Standard
-\noindent
-This flag will increase the time for the cmake command by several seconds,
- mainly because of the process of inverting tests (see Section
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "subsec:Interpreting-export-tests"
-
\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
-).
-\end_layout
+\begin_layout Plain Layout
-\begin_layout Subsubsection
-Running the tests
\end_layout
-\begin_layout Standard
-To run all tests, in the build directory simply run the command
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
+Ignored
\end_layout
\end_inset
-
-.
- To run only some of the tests, use the command
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest -R <pattern>
+Unreliable
\end_layout
\end_inset
-
-, where
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-<pattern>
+Suspicious
\end_layout
\end_inset
-
- is a regular expression that matches test names.
- To run only the export tests, you can use
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest -L export
+Suspended
\end_layout
\end_inset
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-.
- For the list of test categories available in addition to
-\begin_inset Quotes eld
-\end_inset
-
+\begin_layout Plain Layout
export
-\begin_inset Quotes erd
-\end_inset
+\end_layout
-, run
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest \SpecialChar nobreakdash
-\SpecialChar nobreakdash
-print-labels
+
\end_layout
\end_inset
-
-.
- It is often useful to list the tests without running them (e.g.
- if you want to know how many tests there are or whether your
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-<pattern>
+unreliable
\end_layout
\end_inset
-
- regular expression did what you expected).
- This can be done with the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
--N
+
\end_layout
\end_inset
-
- or
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-\SpecialChar nobreakdash
-\SpecialChar nobreakdash
-show-only
+inverted
\end_layout
\end_inset
-
- argument.
- We are still working on getting the tests to run in parallel which is supported
- by the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
+
\end_layout
\end_inset
-
- command with the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
--j <jobs>
+suspended
\end_layout
\end_inset
-
- or
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-\SpecialChar nobreakdash
-\SpecialChar nobreakdash
-parallel <jobs>
+
\end_layout
\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- argument.
- However, when running the tests in parallel, sometimes tests fail that
- pass when run sequentially.
- A reasonable approach is to first run the tests in parallel and then run
- the failed tests sequentially.
- For example, to run 8 jobs at a time:
+\begin_layout Plain Layout
+Yes
\end_layout
-\begin_layout Standard
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest -j8
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest \SpecialChar nobreakdash
-\SpecialChar nobreakdash
-rerun-failed
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-\noindent
-Note that some tests cannot be run in parallel.
- These tests are marked in the code with the
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-\noindent
-RUN_SERIAL ON
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- CMake property.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-In some situations the option
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-\SpecialChar nobreakdash
-\SpecialChar nobreakdash
-timeout <seconds>
+
\end_layout
\end_inset
-
- is useful.
- There have been bugs in LyX and in LaTeX which cause compilation to hang,
- and without a timeout a test might never stop (in one case there was even
- a memory leak).
- If a test times out, the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
+-
\end_layout
\end_inset
-
- command exits with error, but you can distinguish between a timed out test
- and a failed test in the output reported at the end of the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- command.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-See the manual (
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-man ctest
+-
\end_layout
\end_inset
+</cell>
+</row>
+<row>
+<cell multirow="3" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-) the full list of command line options.
+\begin_layout Plain Layout
+No
\end_layout
-\begin_layout Subsubsection
-\begin_inset CommandInset label
-LatexCommand label
-name "subsec:Interpreting-export-tests"
-
\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-Interpreting the export test results
+\begin_layout Plain Layout
+Yes
\end_layout
-\begin_layout Standard
-A test can fail for several reasons, not all of them bad.
-\end_layout
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-\begin_layout Enumerate
-The .lyx file could have been added recently and some formats are not expected
- to work well.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Enumerate
-A dependency is not met (e.g.
- the LaTeX class file).
- One hint that this is the case is that the corresponding
-\begin_inset Flex Code
-status open
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-check_load
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- test will likely also fail.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Enumerate
-An inverted test fails to fail (i.e.
- export that previously failed now works).
-\end_layout
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
-\begin_layout Enumerate
-An external dependency was updated (e.g.
- TeX Live).
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Enumerate
-A recent code change introduced a bug.
-\end_layout
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-\begin_layout Enumerate
-\begin_inset CommandInset label
-LatexCommand label
-name "enu:exposed"
+\begin_layout Plain Layout
+-
+\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
-A change in a document exposed a previously unknown bug or an incompatibility
- with an export format (e.g.
- LuaTeX).
+\begin_layout Plain Layout
++
\end_layout
-\begin_layout Standard
-Because the .lyx files are exported in several formats, it is not surprising
- that many of the exports fail.
- This expectation of failure is addressed by
-\begin_inset Quotes eld
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-inverting
-\begin_inset Quotes erd
-\end_inset
+\begin_layout Plain Layout
+-
+\end_layout
- the tests, that is, by marking the test as
-\begin_inset Quotes eld
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
-passing
-\begin_inset Quotes erd
-\end_inset
+\begin_layout Plain Layout
+-
+\end_layout
- if the export exits with error and as
-\begin_inset Quotes eld
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+-
+\end_layout
-failing
-\begin_inset Quotes erd
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
- if the export succeeds
-\emph on
-.
+\begin_layout Plain Layout
+-
+\end_layout
-\emph default
- It follows that these expected failures will not show up as failed tests
- in the test results and thus will not pollute the
-\begin_inset Quotes eld
\end_inset
+</cell>
+</row>
+<row>
+<cell multirow="4" alignment="left" valignment="middle" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-good
-\begin_inset Quotes erd
-\end_inset
+\begin_layout Plain Layout
- tests.
- If the export actually succeeds, then the test will fail.
- The purpose of this failure is to get your attention—something has changed,
- possibly for the better.
\end_layout
-\begin_layout Standard
-We try to document why a test is inverted or ignored.
- See the comment (prefixed with
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell multirow="3" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-#
+No
\end_layout
\end_inset
+</cell>
+<cell multirow="3" alignment="left" valignment="middle" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-) above the block in which the test is listed as inverted or ignored in
- the files
-\begin_inset Flex Code
-status collapsed
+\begin_layout Plain Layout
+Yes
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-development/autotests/revertedTests
+Yes
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- and
-\begin_inset Flex Code
-status collapsed
+\begin_layout Plain Layout
+-
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-development/autotests/ignoredTests
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-.
- It is possible that an export goes from succeeding to failing just because
- the document was changed and the document was never expected to work with
- a certain export format in the first case.
- Once this is confirmed to be the situation, the test can be inverted.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-A good question is why do we enable the tests for non-default formats? The
- answer is that if a non-default route is broken it is often because a bug
- was introduced in LyX and not because a document-specific change was made
- that is not supported by the route.
- In other words, there is a high signal/noise ratio in the export tests
- for some non-default formats.
-
-\end_layout
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
-\begin_layout Standard
-What action should you take if a test fails? First, check manually that
- when the compilation succeeded before the resulting PDF was good.
- In fact, sometimes it is an improvement when a test fails.
- If you check manually, it might be the case that the export was succeeding
- before but showing garbled text in a PDF output.
- Now it might fail with a clear message of "language xyz not supported".
- It is always good to check manually why something fails and if it passes
- if the PDF output is good.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-Sometimes a test is fixed as side-effect of some change.
- We should uninvert a test (remove it from the
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-revertedTests
++
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
- file) in order to preserve the fix.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-When a test or several tests fail, consider checking the files in the
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-Testing/Temporary/
+-
\end_layout
\end_inset
-
- subdirectory of your build directory.
- In this subdirectory are three files: the file
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-LastTestsFailed.log
++
\end_layout
\end_inset
-
- simply lists the tests that failed on your last
-\begin_inset Flex Code
-status collapsed
+</cell>
+</row>
+<row>
+<cell multirow="4" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
+
\end_layout
\end_inset
-
- command; the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multirow="4" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-LastTest.log
+
\end_layout
\end_inset
-
- file contains the output from the tests (and often has details explaining
- why a test failed); and the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multirow="4" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-CTestCostData.txt
+
\end_layout
\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- file lists the times that it took to run the tests.
+\begin_layout Plain Layout
+No
\end_layout
-\begin_layout Subsubsection
-Inverted tests
-\end_layout
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-\begin_layout Standard
-These tests are expected to always fail.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Description
-reverted These tests are expected to fail, but are subject to be examined.
- It is expected that they will pass in a foreseeable future.
- They are labeled 'reverted'.
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
++
\end_layout
-\begin_layout Description
-suspended Some inverted tests are labeled 'suspended'.
- This means, they are not executed using
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest -L export
+-
\end_layout
\end_inset
-
- or
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest -L reverted
+-
\end_layout
\end_inset
-
-.
- From time to time they still have to be checked using
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest -L suspended
++
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
-.
+\begin_layout Plain Layout
++
\end_layout
-\begin_deeper
-\begin_layout Standard
-These tests are suspended, because they fail for known reasons which cannot
- ATM be resolved.
- But it is expected the reason might disappear in the future.
- Be it new TL or better handling in \SpecialChar LyX
-.
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-For ctest commands without the
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
--L
+-
\end_layout
\end_inset
+</cell>
+</row>
+<row>
+<cell multirow="4" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- parameter nothing changes.
- Suspended or not, tests will be executed depending only on the regexes
- parameters given to the ctest command.
-\end_layout
+\begin_layout Plain Layout
-\end_deeper
-\begin_layout Subsubsection
-Unreliable tests
\end_layout
-\begin_layout Description
-nonstandard Requires non-standard ressources (LaTeX packages and document
- classes, fonts, ...) that are not a requirement for running this test suite.
-\end_layout
+\end_inset
+</cell>
+<cell multirow="4" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
-\begin_deeper
-\begin_layout Standard
-These tests are labeled as
-\family typewriter
-'nonstandard'.
\end_layout
-\end_deeper
-\begin_layout Description
-erratic Tests with
-\begin_inset Quotes eld
\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+No
+\end_layout
-arbitrary
-\begin_inset Quotes erd
\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- result, depending on local configuration, OS, TeX distribution, package
- versions, or the phase of the moon.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_deeper
-\begin_layout Standard
-These tests are labeled as
-\family typewriter
-'erratic'.
-\end_layout
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-\end_deeper
-\begin_layout Subsection
-Export test filtering
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-There are files which control the assignment of a label to a test.
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
++
\end_layout
-\begin_layout Description
-ignoredTests (smal file)
-\begin_inset Newline newline
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-Tests selected here are withdrawn
+\begin_layout Plain Layout
+-
\end_layout
-\begin_deeper
-\begin_layout Labeling
-\labelwidthstring 00.00.0000
-Input Test of any export combination
-\end_layout
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" usebox="none">
+\begin_inset Text
-\begin_layout Labeling
-\labelwidthstring 00.00.0000
-Output Stop if tests not selected here
+\begin_layout Plain Layout
+-
\end_layout
-\end_deeper
-\begin_layout Description
-unreliableTests: Tests selected either pass or fail, but that is dependent
- on the system where the test is run.
- Selected tests gain the label 'unreliable'.
-\end_layout
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-\begin_deeper
-\begin_layout Labeling
-\labelwidthstring 00.00.0000
-Input Each test which passed 'ignoredTests'
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Labeling
-\labelwidthstring 00.00.0000
-Output Stop if test selected, gain label 'unreliable'.
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+-
\end_layout
-\end_deeper
-\begin_layout Description
-suspiciousTests
-\begin_inset space \space{}
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-
+\begin_layout Plain Layout
+-
\end_layout
-\begin_deeper
-\begin_layout Labeling
-\labelwidthstring 00.00.0000
-Input Each test which passed 'unreliableTests'
-\end_layout
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" rightline="true" usebox="none">
+\begin_inset Text
-\begin_layout Labeling
-\labelwidthstring 00.00.0000
-Output Stop if not selected.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-The following file is meant as subselections of 'suspiciousTests'.
- If neither subselection applies, test gains labels 'export' and 'reverted'
-\end_layout
+\end_inset
+</cell>
+</row>
+</lyxtabular>
-\begin_layout Description
-suspendedTests Tests selected here gain the label 'suspended' but _not_
- 'export' or 'reverted'.
-\end_layout
+\end_inset
-\begin_deeper
-\begin_layout Labeling
-\labelwidthstring 00.00.0000
-Input Each test selected by 'suspiciousTests'
-\end_layout
-\begin_layout Labeling
-\labelwidthstring 00.00.0000
-Output Selected test gains label 'suspended'.
-
\end_layout
-\end_deeper
-\end_deeper
-\begin_layout Subsection
+\begin_layout Subsubsection
check_load tests
\end_layout
Under cmake, the tests are labeled as 'load'.
\end_layout
-\begin_layout Subsection
-URL tests (cmake only)
+\begin_layout Subsubsection
+Keytests
+\end_layout
+
+\begin_layout Standard
+Automated tests based on the "MonKey Testing" keytest program.
+ They are documented in the README document in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests
+\end_layout
+
+\end_inset
+
+ subfolder of the \SpecialChar LyX
+ source code distribution.
+ T
+\end_layout
+
+\begin_layout Subsubsection
+lyx21 tests
+\end_layout
+
+\begin_layout Standard
+These tests combine lyx2lyx tests with check_load tests.
+ They fail if either fails.
+\end_layout
+
+\begin_layout Subsubsection
+URL tests
\end_layout
\begin_layout Standard
'url'.
\end_layout
-\begin_layout Subsubsection
+\begin_layout Paragraph
Running URL tests
\end_layout
\begin_inset Newline newline
\end_inset
-runns all tests with label
+runs all tests with label
\family typewriter
'url'
\end_layout
\begin_inset Newline newline
\end_inset
-runns the tests 'check_accessible_urls'
+runs the tests 'check_accessible_urls'
\end_layout
\begin_layout Standard
of the form 'LastFailed.*URLS.log'
\end_layout
-\begin_layout Subsection
-Test labels (cmake only)
-\end_layout
-
-\begin_layout Standard
-ctest label commands:
-\end_layout
-
-\begin_layout Description
-\SpecialChar nobreakdash
-\SpecialChar nobreakdash
-print-labels shows all assigned labels
-\end_layout
-
-\begin_layout Description
-\SpecialChar nobreakdash
-L
-\begin_inset space ~
-\end_inset
-
-labelname executes all tests to which this label is asigned to.
- A test may have more that one label.
-\end_layout
-
-\begin_layout Description
-\SpecialChar nobreakdash
-j
-\begin_inset space ~
-\end_inset
-
-number executes tests in parallel using 'number' simultaneously processes.
- Some tests are marked as 'sequencial', for them this parameter has no effect.
-\end_layout
-
\begin_layout Section
Development policies
\end_layout
projects / lyx.git / blobdiff