\lyxformat 503
\begin_document
\begin_header
-\origin /systemlyxdir/doc/
+\origin /systemlyxdir/doc
\textclass scrartcl
\options BCOR8mm,captions=tableheading
\use_default_options false
\font_sf_scale 100 100
\font_tt_scale 100 100
\graphics default
-\default_output_format default
+\default_output_format pdf2
\output_sync 0
\bibtex_command default
\index_command default
.
\end_layout
+\begin_layout Standard
+Note that we do not update the local layout of our
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+.lyx
+\end_layout
+
+\end_inset
+
+ files because users would then not be able to export to older formats.
+ For example, if a 2.2.0 user exported a template to 2.1.x format and tried
+ to open the file in LyX 2.1.x, there would be an error because the file would
+ contain a local layout whose format is too new.
+ The root reason for this is that we do not support converting layouts to
+ older layout formats, as we do for the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+.lyx
+\end_layout
+
+\end_inset
+
+ file format.
+\end_layout
+
\begin_layout Subsection
Updating the file format number of bind/ui files
\end_layout
\end_layout
\begin_layout Subsection
-automatic LyX tests (cmake only)
+ctest automatic tests (cmake only)
\end_layout
\begin_layout Standard
\end_layout
\begin_layout Standard
-The export tests require the
-\begin_inset Flex Code
-status collapsed
-
-\begin_layout Plain Layout
-ctest
-\end_layout
-
+The export tests
+\begin_inset Quotes eld
\end_inset
- command that comes with the
-\begin_inset Flex Code
-status collapsed
-
-\begin_layout Plain Layout
-cmake
-\end_layout
-
+reuse
+\begin_inset Quotes erd
\end_inset
- build system
+ documentation, template, and example files trying to export them to all
+ supported output formats.
\end_layout
\begin_layout Paragraph
export code (e.g.
touching the encoding code or package handling code that you expect will
change the exported \SpecialChar LaTeX
- in some way), consider running all of the export tests
- before and after your change.
+ in some way):
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+\paragraph_spacing single
+Consider running all of the export tests before and after your change.
If there are differences, please reconcile these (i.e.
fix the bug or fix the tests)
\emph on
before
\emph default
committing.
- Ask for help if you're not sure what to do or if you do not want to run
- the tests, post the patch on the list and others will run the tests.
+ Ask for help if you're not sure what to.
+\end_layout
+
+\begin_layout Standard
+If you do not want to run the tests,
\end_layout
+\begin_layout Itemize
+post the patch on the list and others will run the tests and eventually
+ ask for fixes, or
+\end_layout
+
+\begin_layout Itemize
+commit, but be prepared to fix eventually arising problems or to revert
+ the commit if there is no easy fix.
+\end_layout
+
+\end_deeper
\begin_layout Itemize
Understand how to interpret test failures.
If your commit is found to have broken a test, you should be able to interpret
\end_layout
\begin_layout Standard
-To enable these tests, add the
+To enable the export autotests, add the
\begin_inset Flex Code
status collapsed
\end_inset
.
- To run only some of the tests, use the command
+
+\end_layout
+
+\begin_layout Standard
+To run only some of the tests, use command line options:
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
+-R <pattern>
\end_layout
\end_inset
-, where
+ Run only the tests whose names match the given regular expression.
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-<pattern>
\end_layout
\end_inset
- is a regular expression that matches test names.
- To run only the export tests, you can use
+ Run only the tests whose labels match the given regular expression.
+ A test may have more that one label.
+
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+For example, to run only the export tests, you can use
\begin_inset Flex Code
status collapsed
\end_inset
.
- For the list of test categories available in addition to
-\begin_inset Quotes eld
-\end_inset
-
-export
-\begin_inset Quotes erd
-\end_inset
+
+\end_layout
-, run
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring -R
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-ctest \SpecialChar nobreakdash
-\SpecialChar nobreakdash
-print-labels
+-E <pattern>
\end_layout
\end_inset
-.
- It is often useful to list the tests without running them (e.g.
- if you want to know how many tests there are or whether your
+ Exclude the tests whose names match the given regular expression.
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-<pattern>
\end_layout
\end_inset
- regular expression did what you expected).
- This can be done with the
+ Exclude the tests whose labels match the given regular expression.
+\end_layout
+
+\begin_layout Standard
+The following options help to find good selection patterns:
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
\begin_inset Flex Code
status collapsed
\end_inset
- or
+ List the tests that would be run but not actually run them.
+
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+Useful in conjunction with the -R, -L, -E and -LE options, e.g., if you want
+ to know how many tests there are or whether your
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-\SpecialChar nobreakdash
-\SpecialChar nobreakdash
-show-only
+<pattern>
\end_layout
\end_inset
- argument.
- We are still working on getting the tests to run in parallel which is supported
- by the
+ regular expression did what you expected.
+\end_layout
+
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring -R
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-ctest
+\SpecialChar nobreakdash
+\SpecialChar nobreakdash
+print-labels
\end_layout
\end_inset
- command with the
-\begin_inset Flex Code
-status collapsed
-
-\begin_layout Plain Layout
--j <jobs>
+ print the list of all labels associated with the test set.
+ Can also be combined with -R, -L, -E, ...
+
\end_layout
-\end_inset
+\begin_layout Standard
+Other relevant options are:
+\end_layout
- or
+\begin_layout Labeling
+\labelwidthstring -R
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-\SpecialChar nobreakdash
-\SpecialChar nobreakdash
-parallel <jobs>
+-j <jobs>
\end_layout
\end_inset
- argument.
+ Run the tests in parallel using the given number of jobs.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+We are still working on getting the tests to run in parallel.
However, when running the tests in parallel, sometimes tests fail that
pass when run sequentially.
A reasonable approach is to first run the tests in parallel and then run
the failed tests sequentially.
- For example, to run 8 jobs at a time:
+
+\end_layout
+
+\begin_layout Standard
+For example, to run 8 jobs at a time:
\end_layout
\begin_layout Standard
CMake property.
\end_layout
-\begin_layout Standard
-In some situations the option
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring -R
\begin_inset Flex Code
status collapsed
\end_inset
- is useful.
- There have been bugs in LyX and in \SpecialChar LaTeX
+ Set a global timeout on all tests that do not already have a timeout set
+ on them.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+There have been bugs in LyX and in \SpecialChar LaTeX
which cause compilation to hang, and
without a timeout a test might never stop (in one case there was even a
memory leak).
command.
\end_layout
+\end_deeper
\begin_layout Standard
See the manual (
\begin_inset Flex Code
class file).
One hint that this is the case is that the corresponding
\begin_inset Flex Code
-status open
+status collapsed
\begin_layout Plain Layout
check_load
\end_layout
\begin_layout Standard
-What action should you take if a test fails? First, check manually that
- when the compilation succeeded before the resulting PDF was good.
- In fact, sometimes it is an improvement when a test fails.
- If you check manually, it might be the case that the export was succeeding
- before but showing garbled text in a PDF output.
+What action should you take if a test fails? This depends:
+\end_layout
+
+\begin_layout Standard
+Generally, if a change breaks compilation for the target format (for the
+ manuals pdf2) without solving some important other issue, fix or revert
+ the commit that led to failure.
+ If a change breaks compilation for some non-target format (for the manuals
+ everything except pdf2), invert the test.
+
+\end_layout
+
+\begin_layout Standard
+A special case is given, if the export was succeeding before but showing
+ garbled text in the PDF output.
+ Try to establish, that when the compilation succeeded before the resulting
+ PDF was good.
+ Otherwise, it is in fact an improvement when a test fails.
Now it might fail with a clear message of "language xyz not supported".
It is always good to check manually why something fails and if it passes
if the PDF output is good.
Unreliable tests
\end_layout
+\begin_layout Standard
+These tests are not executed using
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L export
+\end_layout
+
+\end_inset
+
+ or
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L inverted
+\end_layout
+
+\end_inset
+
+.
+
+\end_layout
+
+\begin_layout Standard
+They pass or fail for various reasons not related to LyX (nonstandard, erratic)
+ or pass but should rather fail (wrong output).
+\end_layout
+
\begin_layout Description
-nonstandard Requires non-standard ressources (\SpecialChar LaTeX
- packages and document classes,
- fonts, ...) that are not a requirement for running this test suite.
+nonstandard Documents with additional requirements, e.g.
+ a class or package file not on CTAN.
+
+\begin_inset Note Note
+status open
+
+\begin_layout Plain Layout
+TODO: rename to "extra"?
+\end_layout
+
+\end_inset
+
+
\end_layout
\begin_deeper
\end_deeper
\begin_layout Description
-erratic Tests with
-\begin_inset Quotes eld
-\end_inset
+erratic Tests depending on local configuration, OS, TeX distribution, package
+ versions, or the phase of the moon.
+
+\begin_inset Note Note
+status open
+
+\begin_layout Plain Layout
+TODO: use
+\emph on
+erratic
+\emph default
+ only for the phase-of-moon dependency.
+\end_layout
-arbitrary
-\begin_inset Quotes erd
\end_inset
- result, depending on local configuration, OS, \SpecialChar TeX
- distribution, package versions,
- or the phase of the moon.
+
\end_layout
\begin_deeper
'erratic'.
\end_layout
+\end_deeper
+\begin_layout Description
+wrong
+\begin_inset space ~
+\end_inset
+
+output Export does not fail but the resulting document has errors.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+\paragraph_spacing single
+These tests are actually not
+\emph on
+unreliable
+\emph default
+ but
+\emph on
+invalid
+\emph default
+ (not measuring what they should measure).
+\end_layout
+
\end_deeper
\begin_layout Paragraph
Export test filtering
\end_layout
\begin_layout Description
-ignoredTests (smal file)
+ignoredTests (small file)
\begin_inset Newline newline
\end_inset
projects / lyx.git / blobdiff