#LyX 2.2 created this file. For more info see http://www.lyx.org/
-\lyxformat 499
+\lyxformat 504
\begin_document
\begin_header
+\save_transient_properties true
\origin /systemlyxdir/doc/
-\textclass scrbook
-\options fleqn,bibliography=totoc,index=totoc,BCOR7.5mm,titlepage,captions=tableheading
+\textclass scrartcl
+\options BCOR8mm,captions=tableheading
\use_default_options false
\begin_modules
logicalmkup
-theorems-ams
-theorems-ams-extended
-multicol
-shapepar
\end_modules
\maintain_unincluded_children false
-\begin_local_layout
-Format 7
-InsetLayout CharStyle:MenuItem
-LyxType charstyle
-LabelString menu
-LatexType command
-LatexName menuitem
-Font
-Family Sans
-EndFont
-Preamble
-\newcommand*{\menuitem}[1]{{\sffamily #1}}
-EndPreamble
-End
-\end_local_layout
\language english
\language_package default
\inputencoding auto
\fontencoding global
-\font_roman default
-\font_sans default
-\font_typewriter default
-\font_math auto
+\font_roman "lmodern" "default"
+\font_sans "lmss" "default"
+\font_typewriter "lmtt" "default"
+\font_math "auto" "auto"
\font_default_family default
\use_non_tex_fonts false
\font_sc false
\font_osf false
-\font_sf_scale 100
-\font_tt_scale 100
+\font_sf_scale 100 100
+\font_tt_scale 100 100
\graphics default
-\default_output_format default
+\default_output_format pdf2
\output_sync 0
\bibtex_command default
\index_command default
\shortcut idx
\color #008000
\end_index
-\secnumdepth 3
-\tocdepth 3
+\secnumdepth 4
+\tocdepth 4
\paragraph_separation indent
\paragraph_indentation default
\quotes_language english
\end_layout
-\begin_layout Chapter
+\begin_layout Section
Introduction
\end_layout
source code distribution.
This document is not translated, since the development language of \SpecialChar LyX
is
- english.
+ English.
If you want to use \SpecialChar LyX
you don't need to read this manual.
However, if you want to learn more about how \SpecialChar LyX
development, you may find some interesting information.
\end_layout
-\begin_layout Chapter
+\begin_layout Section
File formats
\end_layout
tion.
\end_layout
-\begin_layout Section
+\begin_layout Subsection
File Format Numbers
\end_layout
-\begin_layout Section
+\begin_layout Subsection
When is an update of the .lyx file format number needed?
\begin_inset CommandInset label
LatexCommand label
package Any new math package that is automatically loaded needs a file format
update.
The reason for this is that there is no true ERT inset for math formulas:
- Each command is parsed, and if a user happens to defne a local command
+ Each command is parsed, and if a user happens to define a local command
with the same name as a command that triggers an automatic load of a package,
he needs to be able to switch off the automatic loading of that package.
This switch is stored by the
If you are still unsure, please ask on the development list.
\end_layout
-\begin_layout Section
+\begin_layout Subsection
How to update the file format number of .lyx files
+\begin_inset CommandInset label
+LatexCommand label
+name "subsec:update_lyx_files"
+
+\end_inset
+
+
\end_layout
\begin_layout Standard
.
\end_layout
+\begin_layout Enumerate
+Update the range of file formats in the array
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+format_relation
+\end_layout
+
+\end_inset
+
+ in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+lib/lyx2lyx/LyX.py
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
\begin_layout Enumerate
Add an entry to both format lists (for conversion and reversion) in
\begin_inset Newline newline
\begin_layout Enumerate
It would be nice if you could create a .lyx test file which contains instances
of all changed or added features.
- This could then be used to test lyxl2yx and tex2lyx.
+ This could then be used to test lyx2lyx and tex2lyx.
Unfortunately it has not yet been decided how to collect such examples,
so please ask on the development list if you want to create one.
\end_layout
-\begin_layout Section
-Backporting new styles to the stable version
-\end_layout
+\begin_layout Enumerate
+\begin_inset CommandInset label
+LatexCommand label
+name "enu:updatefiles"
-\begin_layout Standard
-Starting with the stable \SpecialChar LyX
- 2.1 branch, there is a mechanism in place to backport
- new styles to the stable version without the need to update the file format.
- The basic idea is that the new style definition is automatically copied
- to the document preamble, so that it can even be used by older minor revisions
- that did not yet include the style.
- To backport a new style to the stable version, the following steps are
- needed:
-\end_layout
+\end_inset
-\begin_layout Enumerate
-Add the line
-\begin_inset Flex Code
+Update LyX's .lyx documentation files to the new format.
+ The developer who makes the change knows best what changes to expect when
+ inspecting the resulting diff.
+ Because of this, you might be able to catch a bug in the lyx2lyx code that
+ updates the format just by taking a quick scan through the large diff that
+ is the result
+\begin_inset Note Note
status collapsed
\begin_layout Plain Layout
-ForceLocal -1
+Another advantage is that if later we suspect a bug in lyx2lyx we can easily
+ see which layout update made an unexpected change by looking at the git
+ log of a .lyx file that suffers the problem.
\end_layout
\end_inset
- to the style definition in the development version.
-\end_layout
-
-\begin_layout Enumerate
-Copy the style definition to the stable version, but use
+.
+ To do this, first make sure that there are no changes to the git repository
+ that you will not want to commit (this is needed because it will be convenient
+ to commit with the command
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-ForceLocal 1
+git commit -a
\end_layout
\end_inset
- instead.
- If needed adjust the format to the one used by the stable version (see
- the customization manual for details of the layout file format).
-\end_layout
-
-\begin_layout Enumerate
-For each update of the style in a later stable version, increase the argument
- of
+).
+ Then run the following command in the root folder of the source:
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-ForceLocal
+python development/tools/updatedocs.py
\end_layout
\end_inset
- by one (in the stable version, the development version should not be touched).
-\end_layout
-
-\begin_layout Standard
-For details about the
+.
+ Then, revert the change to
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-ForceLocal
+LFUNs.lyx
\end_layout
\end_inset
- flag see the customization manual.
- No
+ because that file is meant to be generated separately:
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-lyx2lyx
+git checkout lib/doc/LFUNs.lyx
\end_layout
\end_inset
- support is needed for backported styles: Since the style of the development
- version has an infinite version number, it will always be used.
- Furthermore, since its version number is less than one, the style will
- not be written anymore to the document header for files saved by the new
- version.
-\end_layout
-\begin_layout Chapter
-Tests
-\end_layout
+\begin_inset Note Note
+status collapsed
-\begin_layout Standard
-Automated tests are an important tool to detect bugs and regressions in
- software development.
- Some projects like gcc even require each bug fix to be accompanied by a
- test case for the automatic test suite, that would detect this bug.
- Testing interactive features automatically is of course very hard, but
- core functionality like document import and export can be tested quite
- easily, and some tests of this kind exist.
+\begin_layout Plain Layout
+TODO: this step should be done within updatedocs.py
\end_layout
-\begin_layout Section
-\SpecialChar LyX
- tests
+\end_inset
+
+.
+ Look at the resulting changes using the command
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+git diff
\end_layout
-\begin_layout Standard
-Some tests are located in the
+\end_inset
+
+.
+ If anything looks surprising, please investigate.
+ Finally, commit using
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-development/autotests
+git commit -a
\end_layout
\end_inset
- subfolder of the \SpecialChar LyX
- source code distribution.
+.
\end_layout
\begin_layout Subsection
-Running the tests
+Updating the file format number of layout files
\end_layout
\begin_layout Standard
-cmake is required to run the \SpecialChar LyX
- tests, running them is not implemented for
- autotools.
- The \SpecialChar LyX
- tests can be run by the commands
-\begin_inset Flex Code
-status collapsed
+See step
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "enu:updatefiles"
-\begin_layout Plain Layout
-ctest
-\end_layout
+\end_inset
+
+ in section
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "subsec:update_lyx_files"
\end_inset
- (all platforms) or
+ but instead of the
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-make test
+updatedocs.py
\end_layout
\end_inset
- (when using a make based build system and not MSVC) in the
+ command, use this command:
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-autotests
+python development/tools/updatelayouts.py
\end_layout
\end_inset
- subfolder of the build directory.
-\end_layout
-
-\begin_layout Section
-tex2lyx tests
+.
\end_layout
\begin_layout Standard
-The tex2lyx tests are located in the
+Note that we do not update the local layout of our
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-src/tex2lyx/test
+.lyx
\end_layout
\end_inset
- subfolder of the \SpecialChar LyX
- source code distribution.
- The actual testing is performed by the simple python script
+ files because users would then not be able to export to older formats.
+ For example, if a 2.2.0 user exported a template to 2.1.x format and tried
+ to open the file in LyX 2.1.x, there would be an error because the file would
+ contain a local layout whose format is too new.
+ The root reason for this is that we do not support converting layouts to
+ older layout formats, as we do for the
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-src/tex2lyx/test/runtests.py
+.lyx
\end_layout
\end_inset
-.
- Each test consists of two files:
-\begin_inset Flex Code
-status collapsed
+ file format.
+\end_layout
-\begin_layout Plain Layout
-<test name>.tex
+\begin_layout Subsection
+Updating the file format number of bind/ui files
\end_layout
+\begin_layout Standard
+See step
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "enu:updatefiles"
+
\end_inset
- contains the \SpecialChar LaTeX
- code that should be tested.
-
+ in section
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "subsec:update_lyx_files"
+
+\end_inset
+
+ but instead of the
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-<test name>.lyx.lyx
+updatedocs.py
\end_layout
\end_inset
- contains the expected output of tex2lyx.
- When a test is run, the actual produced output is compared with the stored
- reference output.
- The test passes if both are identical.
- The test machinery is also able to generate a file
+ command, use this command:
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-<test name>.lyx.tex
+bash development/tools/updatelfuns.sh
\end_layout
\end_inset
- by exporting the produced .lyx file with \SpecialChar LyX
- again.
- This may be useful for roundtrip comparisons.
+.
\end_layout
\begin_layout Subsection
-Running the tests
+Backporting new styles to the stable version
\end_layout
\begin_layout Standard
-The tex2lyx tests can be run by the commands
-\begin_inset Flex Code
-status collapsed
-
-\begin_layout Plain Layout
-ctest
+Starting with the stable \SpecialChar LyX
+ 2.1 branch, there is a mechanism in place to backport
+ new styles to the stable version without the need to update the file format.
+ The basic idea is that the new style definition is automatically copied
+ to the document preamble, so that it can even be used by older minor revisions
+ that did not yet include the style.
+ To backport a new style to the stable version, the following steps are
+ needed:
\end_layout
-\end_inset
-
- (cmake, all platforms),
+\begin_layout Enumerate
+Add the line
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-make test
+ForceLocal -1
\end_layout
\end_inset
- (cmake, when using a make based build system and not MSVC) or
+ to the style definition in the development version.
+\end_layout
+
+\begin_layout Enumerate
+Copy the style definition to the stable version, but use
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-make alltests
+ForceLocal 1
\end_layout
\end_inset
- (autotools) in the
+ instead.
+ If needed adjust the format to the one used by the stable version (see
+ the customization manual for details of the layout file format).
+\end_layout
+
+\begin_layout Enumerate
+For each update of the style in a later stable version, increase the argument
+ of
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-src/tex2lyx
-\end_layout
-
-\end_inset
-
- subfolder of the build directory.
- If a test fails, the differences between the expected and actual results
- are output in unified diff format.
+ForceLocal
\end_layout
-\begin_layout Subsection
-Updating test references
-\begin_inset CommandInset label
-LatexCommand label
-name "sec:Updating-test-references"
-
\end_inset
-
+ by one (in the stable version, the development version should not be touched).
\end_layout
\begin_layout Standard
-In some cases a changed tex2lyx output is not a test failure, but wanted,
- e.
-\begin_inset space \thinspace{}
-\end_inset
-
-g.
-\begin_inset space \space{}
-\end_inset
-
-if a tex2lyx bug was fixed, or a new feature was added.
- In these cases the stored references need to be updated.
- To do so if using autotools, call
+For details about the
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-make updatetests
+ForceLocal
\end_layout
\end_inset
- in the
+ flag see the customization manual.
+ No
\begin_inset Flex Code
status collapsed
\begin_layout Plain Layout
-src/tex2lyx
+lyx2lyx
+\end_layout
+
+\end_inset
+
+ support is needed for backported styles: Since the style of the development
+ version has an infinite version number, it will always be used.
+ Furthermore, since its version number is less than one, the style will
+ not be written anymore to the document header for files saved by the new
+ version.
+\end_layout
+
+\begin_layout Standard
+\begin_inset Newpage newpage
+\end_inset
+
+
+\end_layout
+
+\begin_layout Section
+Tests
+\end_layout
+
+\begin_layout Standard
+Automated tests are an important tool to detect bugs and regressions in
+ software development.
+ Some projects like gcc even require each bug fix to be accompanied by a
+ test case for the automatic test suite, that would detect this bug.
+ Testing interactive features automatically is of course very hard, but
+ core functionality like document import and export can be tested quite
+ easily, and some tests of this kind exist.
+\end_layout
+
+\begin_layout Subsection
+unit tests
+\end_layout
+
+\begin_layout Standard
+There are attempts to set up a suite of unit tests for LyX.
+\end_layout
+
+\begin_layout Standard
+TODO: describe what is done and what is still to do.
+\end_layout
+
+\begin_layout Subsection
+tex2lyx tests
+\end_layout
+
+\begin_layout Standard
+The tex2lyx tests are located in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx/test
+\end_layout
+
+\end_inset
+
+ subfolder of the \SpecialChar LyX
+ source code distribution.
+ The actual testing is performed by the simple python script
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx/test/runtests.py
+\end_layout
+
+\end_inset
+
+.
+ Each test consists of two files:
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+<test name>.tex
+\end_layout
+
+\end_inset
+
+ contains the \SpecialChar LaTeX
+ code that should be tested.
+
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+<test name>.lyx.lyx
+\end_layout
+
+\end_inset
+
+ contains the expected output of tex2lyx.
+ When a test is run, the actual produced output is compared with the stored
+ reference output.
+ The test passes if both are identical.
+ The test machinery is also able to generate a file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+<test name>.lyx.tex
+\end_layout
+
+\end_inset
+
+ by exporting the produced .lyx file with \SpecialChar LyX
+ again.
+ This may be useful for roundtrip comparisons.
+\end_layout
+
+\begin_layout Subsubsection
+Running the tests
+\end_layout
+
+\begin_layout Standard
+The tex2lyx tests can be run in several ways.
+ When in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx
+\end_layout
+
+\end_inset
+
+ subfolder of the build directory, the commands
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ (cmake, all platforms),
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+make test
+\end_layout
+
+\end_inset
+
+ (cmake, when using a make based build system and not MSVC) or
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+make alltests
+\end_layout
+
+\end_inset
+
+ (autotools) will run the tex2lyx tests.
+ Alternatively, in the root of the build directory, the command
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -R tex2lyx
+\end_layout
+
+\end_inset
+
+ runs all tests whose names match the regex
+\begin_inset Quotes eld
+\end_inset
+
+tex2lyx
+\begin_inset Quotes erd
+\end_inset
+
+.
+ Another way to run the tex2lyx tests in the root build directory is to
+ instead use the command
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L '(cmplyx|roundtrip)'
+\end_layout
+
+\end_inset
+
+, which runs all tests categorized with the label
+\begin_inset Quotes eld
+\end_inset
+
+roundtrip
+\begin_inset Quotes erd
+\end_inset
+
+ or
+\begin_inset Quotes eld
+\end_inset
+
+cmplyx
+\begin_inset Quotes erd
+\end_inset
+
+.
+ If a test fails, the differences between the expected and actual results
+ are output in unified diff format.
+\end_layout
+
+\begin_layout Subsubsection
+Updating test references
+\begin_inset CommandInset label
+LatexCommand label
+name "sec:Updating-test-references"
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+In some cases a changed tex2lyx output is not a test failure, but wanted,
+ e.
+\begin_inset space \thinspace{}
+\end_inset
+
+g.
+\begin_inset space \space{}
+\end_inset
+
+if a tex2lyx bug was fixed, or a new feature was added.
+ In these cases the stored references need to be updated.
+ To do so if using autotools, call
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+make updatetests
+\end_layout
+
+\end_inset
+
+ in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx
\end_layout
\end_inset
status collapsed
\begin_layout Plain Layout
-make updatetex2lyxtests
+make updatetex2lyxtests
+\end_layout
+
+\end_inset
+
+ in the build directory or in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx/test
+\end_layout
+
+\end_inset
+
+ subdirectory of the build directory.
+\begin_inset Foot
+status collapsed
+
+\begin_layout Plain Layout
+Note that this is a case where a make target in the build directory can
+ affect the source directory, which might not be advisable.
+\end_layout
+
+\end_inset
+
+ On Windows do the following:
+\end_layout
+
+\begin_layout Itemize
+Assure that the path to the python.exe is in your system PATH variable.
+\end_layout
+
+\begin_layout Itemize
+Double-click on the file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+updatetex2lyxtests.vcxproj
+\end_layout
+
+\end_inset
+
+ in the build directory or in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx/test
+\end_layout
+
+\end_inset
+
+ subdirectory of your build directory.
+\end_layout
+
+\begin_layout Itemize
+In the appearing MSVC program right-click on the project
+\family sans
+updatetex2lyxtests
+\family default
+ in the project explorer and chose
+\family sans
+Create
+\family default
+.
+\end_layout
+
+\begin_layout Standard
+For convenience, these commands also produce re-exported roundtrip .lyx.tex
+ files.
+ Please examine the changed output carefully before committing the changed
+ files to the repository: Since the test machinery does not do a roundtrip
+ test .tex
+\begin_inset Formula $\Rightarrow$
+\end_inset
+
+ .lyx
+\begin_inset Formula $\Rightarrow$
+\end_inset
+
+ .tex, and does not compare the produced dvi or pdf output, it assumes that
+ the stored .lyx reference produces correct output if processed by \SpecialChar LyX
+.
+ There is only one chance to detect wrong output: before committing a new
+ reference.
+ Once it is committed, it is quite difficult to verify whether it is correct.
+\end_layout
+
+\begin_layout Standard
+Please
+\emph on
+do not
+\emph default
+ update the test references by opening them with \SpecialChar LyX
+ or directly running lyx2lyx
+ on them.
+ This would not work, since lyx2lyx and \SpecialChar LyX
+ produce slightly different files
+ regarding insignificant whitespace and line breaks.
+\end_layout
+
+\begin_layout Subsubsection
+Adding a new test
+\end_layout
+
+\begin_layout Standard
+In many cases tests for new features may be added to one of the existing
+ test files, but sometimes this is not possible or not wanted.
+ Then a new test file needs to be added:
+\end_layout
+
+\begin_layout Enumerate
+Create the new file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx/test/<test name>.tex
+\end_layout
+
+\end_inset
+
+ and run tex2lyx in roundtrip mode to produce the file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx/test/<test name>.lyx.lyx
+\end_layout
+
+\end_inset
+
+.
+ This file will be the new reference.
+\end_layout
+
+\begin_layout Enumerate
+Once you confirmed that the tex2lyx output is correct, add the new files
+ to the corresponding lists in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx/test/runtests.py
+\end_layout
+
+\end_inset
+
+,
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx/Makefile.am
+\end_layout
+
+\end_inset
+
+ and
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+src/tex2lyx/test/CMakeLists.txt
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Enumerate
+Commit the changes to the repository, or send a patch to the development
+ list and ask for committing if you do not have commit rights.
+\end_layout
+
+\begin_layout Subsection
+ctest automatic tests (cmake only)
+\end_layout
+
+\begin_layout Standard
+Some tests are located in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests
+\end_layout
+
+\end_inset
+
+ subfolder of the \SpecialChar LyX
+ source code distribution.
+
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+cmake
+\end_layout
+
+\end_inset
+
+ is required to run the automatic \SpecialChar LyX
+ tests, running them is not implemented
+ for autotools.
+
+\end_layout
+
+\begin_layout Standard
+The \SpecialChar LyX
+ tests can be run by the commands
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ (all platforms) or
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+make test
+\end_layout
+
+\end_inset
+
+ (when using a make based build system and not MSVC) in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+autotests
+\end_layout
+
+\end_inset
+
+ subfolder of the build directory.
+\end_layout
+
+\begin_layout Subsubsection
+Export tests
+\end_layout
+
+\begin_layout Standard
+The export tests are integration tests.
+ They take longer to run and are more likely to break than the tex2lyx tests.
+ Nevertheless, they have caught many regressions and without a better alternativ
+e it is important to keep them up-to-date and understand how they work.
+\end_layout
+
+\begin_layout Standard
+The export tests
+\begin_inset Quotes eld
+\end_inset
+
+reuse
+\begin_inset Quotes erd
+\end_inset
+
+ documentation, template, and example files trying to export them to all
+ supported output formats.
+ In addition, there are a number of dedicated sample documents under
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+autotests/export/
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Paragraph
+Expectations of LyX developers
+\end_layout
+
+\begin_layout Standard
+Because the export tests are integration tests and take a long time to run,
+ LyX developers are rarely expected to run all of the tests.
+ Here are some good practices to follow by developers:
+\end_layout
+
+\begin_layout Itemize
+When making a non-trivial change to a .layout file, run the export and layout
+ tests corresponding with that .layout file.
+\end_layout
+
+\begin_layout Itemize
+When making non-trivial changes to a .lyx file, run the export tests correspondin
+g to that .lyx file.
+\end_layout
+
+\begin_layout Itemize
+When making non-trivial changes to LyX's \SpecialChar LaTeX
+ export code (e.g.
+ touching the encoding code or package handling code that you expect will
+ change the exported \SpecialChar LaTeX
+ in some way):
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+\paragraph_spacing single
+Consider running all of the export tests before and after your change.
+ If there are differences, please reconcile these (i.e.
+ fix the bug or fix the tests)
+\emph on
+before
+\emph default
+ committing.
+ Ask for help if you're not sure what to.
+\end_layout
+
+\begin_layout Standard
+If you do not want to run the tests,
+\end_layout
+
+\begin_layout Itemize
+post the patch on the list and others will run the tests and eventually
+ ask for fixes, or
+\end_layout
+
+\begin_layout Itemize
+commit, but be prepared to fix eventually arising problems or to revert
+ the commit if there is no easy fix.
+\end_layout
+
+\end_deeper
+\begin_layout Itemize
+Understand how to interpret test failures.
+ If your commit is found to have broken a test, you should be able to interpret
+ the test results when made aware of them.
+ See Section
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "subsec:Interpreting-export-tests"
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Paragraph
+\begin_inset CommandInset label
+LatexCommand label
+name "par:Configuring-ctests"
+
+\end_inset
+
+Configuring the tests
+\end_layout
+
+\begin_layout Standard
+To enable the export autotests, add the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-DLYX_ENABLE_EXPORT_TESTS=ON
+\end_layout
+
+\end_inset
+
+ flag.
+ For example:
+\end_layout
+
+\begin_layout Standard
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+cmake -DLYX_ENABLE_EXPORT_TESTS=ON /path/to/source
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\noindent
+This flag will increase the time for the cmake command by several seconds,
+ mainly because of the process of inverting tests (see Section
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "subsec:Interpreting-export-tests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_layout Paragraph
+Running the tests
+\end_layout
+
+\begin_layout Standard
+To run all tests, in the build directory simply run the command
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+.
+
+\end_layout
+
+\begin_layout Standard
+To run only some of the tests, use command line options:
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-R <pattern>
+\end_layout
+
+\end_inset
+
+ Run only the tests whose names match the given regular expression.
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-L <pattern>
+\end_layout
+
+\end_inset
+
+ Run only the tests whose labels match the given regular expression.
+ A test may have more that one label.
+
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+For example, to run only the export tests, you can use
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L export
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-E <pattern>
+\end_layout
+
+\end_inset
+
+ Exclude the tests whose names match the given regular expression.
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-LE <pattern>
+\end_layout
+
+\end_inset
+
+ Exclude the tests whose labels match the given regular expression.
+\end_layout
+
+\begin_layout Standard
+The following options help to find good selection patterns:
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-N
+\end_layout
+
+\end_inset
+
+ List the tests that would be run but not actually run them.
+
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+Useful in conjunction with the -R, -L, -E and -LE options, e.g., if you want
+ to know how many tests there are or whether your
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+<pattern>
+\end_layout
+
+\end_inset
+
+ regular expression did what you expected.
+\end_layout
+
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+\SpecialChar nobreakdash
+\SpecialChar nobreakdash
+print-labels
+\end_layout
+
+\end_inset
+
+ print the list of all labels associated with the test set.
+ Can also be combined with -R, -L, -E, ...
+
+\end_layout
+
+\begin_layout Standard
+Other relevant options are:
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-j <jobs>
+\end_layout
+
+\end_inset
+
+ Run the tests in parallel using the given number of jobs.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+We are still working on getting the tests to run in parallel.
+ However, when running the tests in parallel, sometimes tests fail that
+ pass when run sequentially.
+ A reasonable approach is to first run the tests in parallel and then run
+ the failed tests sequentially.
+
+\end_layout
+
+\begin_layout Standard
+For example, to run 8 jobs at a time:
+\end_layout
+
+\begin_layout Standard
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -j8
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest \SpecialChar nobreakdash
+\SpecialChar nobreakdash
+rerun-failed
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\noindent
+Note that some tests cannot be run in parallel.
+ These tests are marked in the code with the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+\noindent
+RUN_SERIAL ON
+\end_layout
+
+\end_inset
+
+ CMake property.
+\end_layout
+
+\end_deeper
+\begin_layout Labeling
+\labelwidthstring -R
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+\SpecialChar nobreakdash
+\SpecialChar nobreakdash
+timeout <seconds>
+\end_layout
+
+\end_inset
+
+ Set a global timeout on all tests that do not already have a timeout set
+ on them.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+There have been bugs in LyX and in \SpecialChar LaTeX
+ which cause compilation to hang, and
+ without a timeout a test might never stop (in one case there was even a
+ memory leak).
+ If a test times out, the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ command exits with error, but you can distinguish between a timed out test
+ and a failed test in the output reported at the end of the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ command.
+\end_layout
+
+\end_deeper
+\begin_layout Standard
+See the manual (
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+man ctest
+\end_layout
+
+\end_inset
+
+) the full list of command line options.
+\end_layout
+
+\begin_layout Paragraph
+\begin_inset CommandInset label
+LatexCommand label
+name "subsec:Interpreting-export-tests"
+
+\end_inset
+
+Interpreting the export test results
+\end_layout
+
+\begin_layout Standard
+A test can fail for several reasons, not all of them bad.
+\end_layout
+
+\begin_layout Enumerate
+The .lyx file could have been added recently and some formats are not expected
+ to work well.
+\end_layout
+
+\begin_layout Enumerate
+A dependency is not met (e.g.
+ the \SpecialChar LaTeX
+ class file).
+ One hint that this is the case is that the corresponding
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+check_load
+\end_layout
+
+\end_inset
+
+ test will likely also fail.
+\end_layout
+
+\begin_layout Enumerate
+An inverted test fails to fail (i.e.
+ export that previously failed now works).
+\end_layout
+
+\begin_layout Enumerate
+An external dependency was updated (e.g.
+ \SpecialChar TeX
+ Live).
+\end_layout
+
+\begin_layout Enumerate
+A recent code change introduced a bug.
+\end_layout
+
+\begin_layout Enumerate
+\begin_inset CommandInset label
+LatexCommand label
+name "enu:exposed"
+
+\end_inset
+
+A change in a document exposed a previously unknown bug or an incompatibility
+ with an export format (e.g.
+ Lua\SpecialChar LaTeX
+).
+\end_layout
+
+\begin_layout Standard
+Because the .lyx files are exported in several formats, it is not surprising
+ that many of the exports fail.
+ This expectation of failure is addressed by
+\begin_inset Quotes eld
+\end_inset
+
+inverting
+\begin_inset Quotes erd
+\end_inset
+
+ the tests, that is, by marking the test as
+\begin_inset Quotes eld
+\end_inset
+
+passing
+\begin_inset Quotes erd
+\end_inset
+
+ if the export exits with error and as
+\begin_inset Quotes eld
+\end_inset
+
+failing
+\begin_inset Quotes erd
+\end_inset
+
+ if the export succeeds
+\emph on
+.
+
+\emph default
+ It follows that these expected failures will not show up as failed tests
+ in the test results and thus will not pollute the
+\begin_inset Quotes eld
+\end_inset
+
+good
+\begin_inset Quotes erd
+\end_inset
+
+ tests.
+ If the export actually succeeds, then the test will fail.
+ The purpose of this failure is to get your attention—something has changed,
+ possibly for the better.
+\end_layout
+
+\begin_layout Standard
+We try to document why a test is inverted or ignored.
+ See the comment (prefixed with
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+#
+\end_layout
+
+\end_inset
+
+) above the block in which the test is listed as inverted or ignored in
+ the files
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/suspiciousTests
+\end_layout
+
+\end_inset
+
+ and
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+development/autotests/ignoredTests
+\end_layout
+
+\end_inset
+
+.
+ It is possible that an export goes from succeeding to failing just because
+ the document was changed and the document was never expected to work with
+ a certain export format in the first case.
+ Once this is confirmed to be the situation, the test can be inverted.
+\end_layout
+
+\begin_layout Standard
+A good question is why do we enable the tests for non-default formats? The
+ answer is that if a non-default route is broken it is often because a bug
+ was introduced in LyX and not because a document-specific change was made
+ that is not supported by the route.
+ In other words, there is a high signal/noise ratio in the export tests
+ for some non-default formats.
+
+\end_layout
+
+\begin_layout Standard
+What action should you take if a test fails? This depends:
+\end_layout
+
+\begin_layout Standard
+Generally, if a change breaks compilation for the target format (for the
+ manuals pdf2) without solving some important other issue, fix or revert
+ the commit that led to failure.
+ If a change breaks compilation for some non-target format (for the manuals
+ everything except pdf2), invert the test.
+
+\end_layout
+
+\begin_layout Standard
+A special case is given, if the export was succeeding before but showing
+ garbled text in the PDF output.
+ Try to establish, that when the compilation succeeded before the resulting
+ PDF was good.
+ Otherwise, it is in fact an improvement when a test fails.
+ Now it might fail with a clear message of "language xyz not supported".
+ It is always good to check manually why something fails and if it passes
+ if the PDF output is good.
+\end_layout
+
+\begin_layout Standard
+Sometimes a test is fixed as side-effect of some change.
+ We should uninvert a test (remove it from the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+suspiciousTests
+\end_layout
+
+\end_inset
+
+ file) in order to preserve the fix.
+\end_layout
+
+\begin_layout Standard
+When a test or several tests fail, consider checking the files in the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+Testing/Temporary/
+\end_layout
+
+\end_inset
+
+ subdirectory of your build directory.
+ In this subdirectory are three files: the file
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+LastTestsFailed.log
+\end_layout
+
+\end_inset
+
+ simply lists the tests that failed on your last
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest
+\end_layout
+
+\end_inset
+
+ command; the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+LastTest.log
+\end_layout
+
+\end_inset
+
+ file contains the output from the tests (and often has details explaining
+ why a test failed); and the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+CTestCostData.txt
+\end_layout
+
+\end_inset
+
+ file lists the times that it took to run the tests.
+\end_layout
+
+\begin_layout Paragraph
+Inverted tests
+\end_layout
+
+\begin_layout Standard
+These tests fail if the export does
+\emph on
+not
+\emph default
+ return an error.
+
+\end_layout
+
+\begin_layout Description
+inverted Export fails, but the test cases are subject to be examined.
+ It is expected that the export will work in a foreseeable future.
+ They are labeled 'inverted'.
+\end_layout
+
+\begin_layout Description
+suspended Some inverted tests are labeled 'suspended'.
+ This means, they are not executed using
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L export
+\end_layout
+
+\end_inset
+
+ or
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L inverted
+\end_layout
+
+\end_inset
+
+.
+ From time to time they still have to be checked using
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L suspended
+\end_layout
+
+\end_inset
+
+.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+These tests are suspended, because the export fails for known reasons which
+ cannot ATM be resolved.
+ But it is expected the reason might disappear in the future.
+ Be it new TL or better handling in \SpecialChar LyX
+.
+\end_layout
+
+\begin_layout Standard
+For ctest commands without the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-L
+\end_layout
+
+\end_inset
+
+ parameter nothing changes.
+ Suspended or not, tests will be executed depending only on the regexes
+ parameters given to the ctest command.
+\end_layout
+
+\end_deeper
+\begin_layout Paragraph
+Unreliable tests
+\end_layout
+
+\begin_layout Standard
+These tests are not executed using
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L export
+\end_layout
+
+\end_inset
+
+ or
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+ctest -L inverted
+\end_layout
+
+\end_inset
+
+.
+
+\end_layout
+
+\begin_layout Standard
+They pass or fail for various reasons not related to LyX (nonstandard, erratic)
+ or pass but should rather fail (wrong output).
+\end_layout
+
+\begin_layout Description
+nonstandard Documents with additional requirements, e.g.
+ a class or package file not on CTAN.
+
+\begin_inset Note Note
+status open
+
+\begin_layout Plain Layout
+TODO: rename to "extra"?
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+These tests are labeled as
+\family typewriter
+'nonstandard'.
+\end_layout
+
+\end_deeper
+\begin_layout Description
+erratic Tests depending on local configuration, OS, TeX distribution, package
+ versions, or the phase of the moon.
+
+\begin_inset Note Note
+status open
+
+\begin_layout Plain Layout
+TODO: use
+\emph on
+erratic
+\emph default
+ only for the phase-of-moon dependency.
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+These tests are labeled as
+\family typewriter
+'erratic'.
+\end_layout
+
+\end_deeper
+\begin_layout Description
+wrong
+\begin_inset space ~
+\end_inset
+
+output Export does not fail but the resulting document has errors.
+\end_layout
+
+\begin_deeper
+\begin_layout Standard
+\paragraph_spacing single
+These tests are actually not
+\emph on
+unreliable
+\emph default
+ but
+\emph on
+invalid
+\emph default
+ (not measuring what they should measure).
+\end_layout
+
+\end_deeper
+\begin_layout Paragraph
+Export test filtering
+\end_layout
+
+\begin_layout Standard
+The assignment of a label to a test is controlled by a set of files with
+ regular expressions that are matched against the test names.
+\end_layout
+
+\begin_layout Description
+ignoredTests (small file)
+\begin_inset Newline newline
+\end_inset
+
+Tests selected here are withdrawn in the configuration step (cf.
+
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "par:Configuring-ctests"
+
+\end_inset
+
+).
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Input Test of any export combination
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Output Stop if tests not selected here
+\end_layout
+
+\end_deeper
+\begin_layout Description
+unreliableTests: Tests selected either pass or fail, but that is dependent
+ on the system where the test is run.
+ Selected tests gain the label 'unreliable'.
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Input Each test which passed 'ignoredTests'
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Output Stop if test selected, gain label 'unreliable'.
+\end_layout
+
+\end_deeper
+\begin_layout Description
+suspiciousTests
+\begin_inset space \space{}
+\end_inset
+
+
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Input Each test which passed 'unreliableTests'
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Output Stop if not selected.
+\end_layout
+
+\begin_layout Standard
+The following file is meant as subselections of 'suspiciousTests'.
+ If neither subselection applies, test gains labels 'export' and 'inverted'
+\end_layout
+
+\begin_layout Description
+suspendedTests Tests selected here gain the label 'suspended' but _not_
+ 'export' or 'inverted', although in ctest they remain inverted.
+ ('ctest' knows only 'inverted' or not, labels are used only for test selection)
+\end_layout
+
+\begin_deeper
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Input Each test selected by 'suspiciousTests'
+\end_layout
+
+\begin_layout Labeling
+\labelwidthstring 00.00.0000
+Output Selected test gains label 'suspended'.
+
+\end_layout
+
+\end_deeper
+\end_deeper
+\begin_layout Standard
+The following table may clarify label assignement
+\end_layout
+
+\begin_layout Standard
+\begin_inset Tabular
+<lyxtabular version="3" rows="7" columns="12">
+<features tabularvalignment="middle">
+<column alignment="left" valignment="top" width="0pt">
+<column alignment="left" valignment="top" width="0pt">
+<column alignment="left" valignment="top" width="0pt">
+<column alignment="left" valignment="top" width="0pt">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<column alignment="center" valignment="top">
+<row>
+<cell multicolumn="1" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+Test found in file:
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+Marked in ctest, Assigned label
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+
\end_layout
\end_inset
-
- in the build directory or in the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-src/tex2lyx/test
+
\end_layout
\end_inset
-
- subdirectory of the build directory
-\begin_inset Foot
-status collapsed
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-Note that this is a case where a make target in the build directory can
- affect the source directory, which might not be advisable.
+
\end_layout
\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
-.
- For convenience, these commands also produce re-exported roundtrip .lyx.tex
- files.
- Please examine the changed output carefully before committing the changed
- files to the repository: Since the test machinery does not do a roundtrip
- test .tex
-\begin_inset Formula $\Rightarrow$
-\end_inset
+\begin_layout Plain Layout
+
+\end_layout
- .lyx
-\begin_inset Formula $\Rightarrow$
\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- .tex, and does not compare the produced dvi or pdf output, it assumes that
- the stored .lyx reference produces correct output if processed by \SpecialChar LyX
-.
- There is only one chance to detect wrong output: before committing a new
- reference.
- Once it is committed, it is quite difficult to verify whether it is correct.
+\begin_layout Plain Layout
+Ignored
\end_layout
-\begin_layout Standard
-Please
-\emph on
-do not
-\emph default
- update the test references by opening them with \SpecialChar LyX
- or directly running lyx2lyx
- on them.
- This would not work, since lyx2lyx and \SpecialChar LyX
- produce slightly different files
- regarding insignificant whitespace and line breaks.
-\end_layout
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-\begin_layout Subsection
-Adding a new test
+\begin_layout Plain Layout
+Unreliable
\end_layout
-\begin_layout Standard
-In many cases tests for new features may be added to one of the existing
- test files, but sometimes this is not possible or not wanted.
- Then a new test file needs to be added:
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+Suspicious
\end_layout
-\begin_layout Enumerate
-Create the new file
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-src/tex2lyx/test/<test name>.tex
+Suspended
\end_layout
\end_inset
-
- and run tex2lyx in roundtrip mode to produce the file
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-src/tex2lyx/test/<test name>.lyx.lyx
+export
\end_layout
\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
-.
- This file will be the new reference.
\end_layout
-\begin_layout Enumerate
-Once you confirmed that the tex2lyx output is correct, add the new files
- to the corresponding lists in
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-src/tex2lyx/test/runtests.py
+unreliable
\end_layout
\end_inset
-
-,
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-src/tex2lyx/Makefile.am
+
\end_layout
\end_inset
-
- and
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-src/tex2lyx/test/CMakeLists.txt
+inverted
\end_layout
\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
-.
-\end_layout
+\begin_layout Plain Layout
-\begin_layout Enumerate
-Commit the changes to the repository, or send a patch to the development
- list and ask for committing if you do not have commit rights.
\end_layout
-\begin_layout Section
-Export tests
-\end_layout
+\end_inset
+</cell>
+<cell multicolumn="1" alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
+\begin_inset Text
-\begin_layout Standard
-The export tests are integration tests.
- They take longer to run and are more likely to break than the tex2lyx tests.
- Nevertheless, they have caught many regressions and without a better alternativ
-e it is important to keep them up-to-date and understand how they work.
+\begin_layout Plain Layout
+suspended
\end_layout
-\begin_layout Subsection
-Expectations of LyX developers
-\end_layout
+\end_inset
+</cell>
+<cell multicolumn="2" alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
-\begin_layout Standard
-Because the export tests are integration tests and take a long time to run,
- LyX developers are rarely expected to run all of the tests.
- Here are some good practices to follow by developers:
-\end_layout
+\begin_layout Plain Layout
-\begin_layout Itemize
-When making a non-trivial change to a .layout file, run the export and layout
- tests corresponding with that .layout file.
\end_layout
-\begin_layout Itemize
-When making non-trivial changes to a .lyx file, run the export tests correspondin
-g to that .lyx file.
-\end_layout
+\end_inset
+</cell>
+</row>
+<row>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-\begin_layout Itemize
-When making non-trivial changes to LyX's LaTeX export code (e.g.
- touching the encoding code or package handling code that you expect will
- change the exported LaTeX in some way), consider running all of the export
- tests before and after your change.
- If there are differences, please reconcile these (i.e.
- fix the bug or fix the tests)
-\emph on
-before
-\emph default
- committing.
- Ask for help if you're not sure what to do or if you do not want to run
- the tests, post the patch on the list and others will run the tests.
+\begin_layout Plain Layout
+Yes
\end_layout
-\begin_layout Itemize
-Understand how to interpret test failures.
- If your commit is found to have broken a test, you should be able to interpret
- the test results when made aware of them.
- See Section
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "subsec:Interpreting-export-tests"
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+-
+\end_layout
\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Subsection
-Configuring the tests
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-To enable these tests when using CMake, add the
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
--DLYX_ENABLE_EXPORT_TESTS=ON
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
- flag.
- For example:
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-cmake -DLYX_ENABLE_EXPORT_TESTS=ON /path/to/source
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
+\begin_layout Plain Layout
\end_layout
-\begin_layout Standard
-\noindent
-This flag will increase the time for the cmake command by several seconds,
- mainly because of the process of inverting tests (see Section
-\begin_inset CommandInset ref
-LatexCommand ref
-reference "subsec:Interpreting-export-tests"
-
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-).
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Subsection
-Running the tests
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-To run all of the export tests, in the build directory simply run the command
-
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
+-
\end_layout
\end_inset
-
-.
- To run only some of the tests, use the command
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest -R <pattern>
+-
\end_layout
\end_inset
-
-, where
-\begin_inset Flex Code
-status collapsed
+</cell>
+</row>
+<row>
+<cell multirow="3" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-<pattern>
+No
\end_layout
\end_inset
-
- is a regular expression that matches test names.
- It is often useful to list the tests without running them (e.g.
- if you want to know how many tests there are or whether your
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-<pattern>
+Yes
\end_layout
\end_inset
-
- regular expression did what you expected).
- This can be done with the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
--N
+-
\end_layout
\end_inset
-
- or
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
---show-only
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- argument.
- We are still working on getting the tests to run in parallel which is supported
- by the
-\begin_inset Flex Code
-status collapsed
+\begin_layout Plain Layout
+-
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- command with the
-\begin_inset Flex Code
-status collapsed
+\begin_layout Plain Layout
+-
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
--j <jobs>
++
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- or
-\begin_inset Flex Code
-status collapsed
+\begin_layout Plain Layout
+-
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
---parallel <jobs>
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- argument.
- However, when running the tests in parallel, sometimes tests fail that
- pass when run sequentially.
- A reasonable approach is to first run the tests in parallel and then run
- the failed tests sequentially.
- For example, to run 8 jobs at a time:
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest -j8
+-
\end_layout
\end_inset
+</cell>
+</row>
+<row>
+<cell multirow="4" alignment="left" valignment="middle" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+\begin_layout Plain Layout
\end_layout
-\begin_layout Standard
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell multirow="3" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest -
-\begin_inset ERT
-status open
+No
+\end_layout
-\begin_layout Plain Layout
+\end_inset
+</cell>
+<cell multirow="3" alignment="left" valignment="middle" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-{}
+\begin_layout Plain Layout
+Yes
\end_layout
\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
--rerun-failed
+\begin_layout Plain Layout
+Yes
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-\noindent
-Note that some tests cannot be run in parallel.
- These tests are marked in the code with the
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-\noindent
-RUN_SERIAL ON
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- CMake property.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-In some situations the option
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
---timeout <seconds>
+-
\end_layout
\end_inset
-
- is useful.
- There have been bugs in LyX and in LaTeX which cause compilation to hang,
- and without a timeout a test might never stop (in one case there was even
- a memory leak).
- If a test times out, the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
++
\end_layout
\end_inset
-
- command exits with error, but you can distinguish between a timed out test
- and a failed test in the output reported at the end of the
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
- command.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Subsection
-\begin_inset CommandInset label
-LatexCommand label
-name "subsec:Interpreting-export-tests"
-
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
-Interpreting the export test results
+\begin_layout Plain Layout
++
\end_layout
-\begin_layout Standard
-A test can fail for several reasons, not all of them bad.
-\end_layout
+\end_inset
+</cell>
+</row>
+<row>
+<cell multirow="4" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
-\begin_layout Itemize
-The .lyx file could have been added recently and some formats are not expected
- to work well.
\end_layout
-\begin_layout Itemize
-A dependency is not met (e.g.
- the LaTeX class file).
- One hint that this is the case is that the corresponding
-\begin_inset Flex Code
-status open
+\end_inset
+</cell>
+<cell multirow="4" alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-check_load
+
\end_layout
\end_inset
+</cell>
+<cell multirow="4" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- test will likely also fail.
-\end_layout
+\begin_layout Plain Layout
-\begin_layout Itemize
-An export that previously failed to compile now compiles.
\end_layout
-\begin_layout Itemize
-An external dependency was updated (e.g.
- TeX Live).
-\end_layout
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-\begin_layout Itemize
-A recent code change introduced a bug.
+\begin_layout Plain Layout
+No
\end_layout
-\begin_layout Standard
-Because the .lyx files are exported in several formats, it is not surprising
- that many of the exports fail.
- This expectation of failure is addressed by
-\begin_inset Quotes eld
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
-inverting
-\begin_inset Quotes erd
-\end_inset
+\begin_layout Plain Layout
+-
+\end_layout
- the tests, that is, by marking the test as
-\begin_inset Quotes eld
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
-passing
-\begin_inset Quotes erd
-\end_inset
+\begin_layout Plain Layout
++
+\end_layout
- if the export exits with error and as
-\begin_inset Quotes eld
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
+-
+\end_layout
-failing
-\begin_inset Quotes erd
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
- if the export succeeds
-\emph on
-.
+\begin_layout Plain Layout
+-
+\end_layout
-\emph default
- It follows that these expected failures will not show up as failed tests
- in the test results and thus will not pollute the
-\begin_inset Quotes eld
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
+
+\begin_layout Plain Layout
++
+\end_layout
-good
-\begin_inset Quotes erd
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" usebox="none">
+\begin_inset Text
- tests.
- If the export actually succeeds, then the test will fail.
- The purpose of this failure is to get your attentions—something has changed,
- possibly for the better.
+\begin_layout Plain Layout
++
\end_layout
-\begin_layout Standard
-We try to document why a test is inverted or ignored.
- See the comment (prefixed with
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-#
+-
\end_layout
\end_inset
-
-) above the block in which the test is listed as inverted or ignored in
- the files
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-development/autotests/revertedTests
+-
\end_layout
\end_inset
-
- and
-\begin_inset Flex Code
-status collapsed
+</cell>
+</row>
+<row>
+<cell multirow="4" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-development/autotests/ignoredTests
+
\end_layout
\end_inset
+</cell>
+<cell multirow="4" alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
-.
-\end_layout
+\begin_layout Plain Layout
-\begin_layout Standard
-What action should you take if a test fails? First, check manually that
- when the compilation succeeded before the resulting PDF was good.
- In fact, sometimes it is an improvement when a test fails.
- If you check manually, it might be the case that the export was succeeding
- before but showing garbled text in a PDF output.
- Now it might fail with a clear message of "language xyz not supported".
- It is always good to check manually why something fails and if it passes
- if the PDF output is good.
\end_layout
-\begin_layout Standard
-Sometimes a test is fixed by accident.
- We should uninvert a test (remove it from the
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-revertedTests
+No
\end_layout
\end_inset
+</cell>
+<cell alignment="left" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- file) in order to preserve the fix.
+\begin_layout Plain Layout
+-
\end_layout
-\begin_layout Standard
-When a test or several tests fail, consider checking the files in the
-\begin_inset Flex Code
-status collapsed
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-Testing/Temporary/
+-
\end_layout
\end_inset
-
- subdirectory of your build directory.
- In this subdirectory are three files: the file
-\begin_inset Flex Code
-status collapsed
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-LastTestsFailed.log
++
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- simply lists the tests that failed on your last
-\begin_inset Flex Code
-status collapsed
+\begin_layout Plain Layout
+-
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-ctest
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- command; the
-\begin_inset Flex Code
-status collapsed
+\begin_layout Plain Layout
+-
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-LastTest.log
+-
\end_layout
\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
+\begin_inset Text
- file contains the output from the tests (and often has details explaining
- why a test failed); and the
-\begin_inset Flex Code
-status collapsed
+\begin_layout Plain Layout
+-
+\end_layout
+
+\end_inset
+</cell>
+<cell alignment="center" valignment="top" topline="true" bottomline="true" rightline="true" usebox="none">
+\begin_inset Text
\begin_layout Plain Layout
-CTestCostData.txt
+-
\end_layout
\end_inset
+</cell>
+</row>
+</lyxtabular>
+
+\end_inset
+
- file lists the times that it took to run the tests.
\end_layout
-\begin_layout Section
+\begin_layout Subsubsection
check_load tests
\end_layout
.
\end_layout
-\begin_layout Section
+\begin_layout Standard
+Under cmake, the tests are labeled as 'load'.
+\end_layout
+
+\begin_layout Subsubsection
+lyx21 tests
+\end_layout
+
+\begin_layout Standard
+These tests combine lyx2lyx tests with check_load tests.
+ They fail if either fails.
+\end_layout
+
+\begin_layout Subsubsection
URL tests
\end_layout
\begin_layout Standard
-These tests are extremely fragile and a failed URL test should not be taken
- too seriously.
- They are useful for finding broken links in our documentation files.
- These must be enabled at configure.
+The URL tests are enabled with the
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+-DLYX_ENABLE_URLTESTS=ON
+\end_layout
+
+\end_inset
+
+ CMake flag and are useful for finding broken links in our documentation
+ files.
+ If a URL test fails, to see which link in particular was reported as broken,
+ see the output in
+\begin_inset Flex Code
+status collapsed
+
+\begin_layout Plain Layout
+LastTest.log
+\end_layout
+
+\end_inset
+
+.
+ These tests are extremely fragile (e.g.
+ a test can depend on your Internet connection) and a failed URL test should
+ not be taken too seriously.
+ URL tests are labeled as
+\family typewriter
+'url'.
+\end_layout
+
+\begin_layout Paragraph
+Running URL tests
+\end_layout
+
+\begin_layout Standard
+cmake is required to run the \SpecialChar LyX
+ tests, running them is not implemented for
+ autotools.
+\end_layout
+
+\begin_layout Standard
+The appropriate commands are:
+\end_layout
+
+\begin_layout Itemize
+
+\family typewriter
+ctest -L url
+\family default
+
+\begin_inset Newline newline
+\end_inset
+
+runs all tests with label
+\family typewriter
+'url'
+\end_layout
+
+\begin_layout Itemize
+
+\family typewriter
+ctest -R 'check_.*urls'
+\family default
+
+\begin_inset Newline newline
+\end_inset
+
+runs the tests 'check_accessible_urls'
+\end_layout
+
+\begin_layout Standard
+Associated test results can be examined in ctest-log directory in files
+ of the form 'LastFailed.*URLS.log'
+\end_layout
+
+\begin_layout Subsubsection
+Test labels
+\end_layout
+
+\begin_layout Standard
+ctest label commands:
+\end_layout
+
+\begin_layout Description
+\SpecialChar nobreakdash
+\SpecialChar nobreakdash
+print-labels shows all assigned labels
+\end_layout
+
+\begin_layout Description
+\SpecialChar nobreakdash
+L
+\begin_inset space ~
+\end_inset
+
+labelname executes all tests to which this label is assigned to.
+ A test may have more that one label.
+\end_layout
+
+\begin_layout Description
+\SpecialChar nobreakdash
+j
+\begin_inset space ~
+\end_inset
+
+number executes tests in parallel using 'number' simultaneously processes.
+ Some tests are marked as 'sequencial', for them this parameter has no effect.
\end_layout
-\begin_layout Chapter
+\begin_layout Section
Development policies
\end_layout
.
\end_layout
-\begin_layout Section
+\begin_layout Subsection
When to set a fixed milestone?
\end_layout
For all other bugs, do not set a milestone at all.
\end_layout
-\begin_layout Section
+\begin_layout Subsection
Can we add rc entries in stable branch?
\end_layout
We are supposed to increase the prefs2prefs version number with such things.
\end_layout
-\begin_layout Chapter
+\begin_layout Section
Documentation policies
\end_layout
projects / lyx.git / blobdiff