X-Git-Url: https://git.lyx.org/gitweb/?a=blobdiff_plain;f=development%2Fautotests%2FunreliableTests;h=7ec725818cdf2098c2c99c9c67c1097f9ecc317c;hb=7a0c8f366d8e8ba1df534c3f5644d789a9c8a6e8;hp=a5c6154eb46e8b051e9330305f30040d1adfa9c4;hpb=3a70340a95339ce3cdabf77232e0e70c2f1053ad;p=lyx.git diff --git a/development/autotests/unreliableTests b/development/autotests/unreliableTests index a5c6154eb4..7ec725818c 100644 --- a/development/autotests/unreliableTests +++ b/development/autotests/unreliableTests @@ -1,60 +1,253 @@ # Regular expressions for tests that should not be executed with -# ctest -L export or ctest -L inverted because they are inapt +# ctest -L export or ctest -L inverted because they do not work as expected # (either unreliable or invalid). # # Patterns will be automatically enclosed with '^' and '$'. -# Matching tests get the label "unreliable". +# Matching tests get the label "unreliable" and do not get the label "export". +# +# TODO: Name problem: *invalid* tests (wrong output) are not *unreliable*. +# Use "unfit" or "unapplicable" as better label and name of pattern file? + + +# These are not unreliable (not dependent on any external driver) +!.*(/ja|)/.*(docbook5|xhtml|lyx[12][0123456]) Sublabel: nonstandard +##################### # Documents with additional requirements, -# e.g. a class or package file not on CTAN. -# TODO: rename to "extra"? +# e.g. a class or package file not in TeXLive. +# TODO: rename to "extra" or "exotic"? + +# aa.cls (not in TeXLive). +export/examples/Articles/Astronomy_%26_Astrophysics.* +export/templates/Articles/Astronomy_%26_Astrophysics.* + +# AEA.cls (see https://wiki.lyx.org/Examples/AEA) +export/templates/Articles/American_Economic_Association_.*AEA.* + +# copernicus.cls (not in TeXLive) +# see https://publications.copernicus.org/for_authors/manuscript_preparation.html +export/templates/Articles/Copernicus_Publications_Manuscript_Preparation_.* + +# ectaart.cls +# see https://wiki.lyx.org/Examples/Econometrica +export/templates/Articles/Econometrica.* + +# iucr.cls (not in TeXLive) +export/templates/Articles/International_Union_of_Crystallography.* + +# ws-ijmpc.cls, w-ijmpd.cls +# see https://wiki.lyx.org/Examples/IJMP +export/templates/Articles/Int._Journal_of_Modern_Physics_C.* +export/templates/Articles/Int._Journal_of_Modern_Physics_D.* + +# iopart.cls, iopams.sty, setstack.sty +# see https://wiki.lyx.org/Layouts/Iopart +export/templates/Articles/Institute_of_Physics_.* + +# jasatex.cls +# see https://wiki.lyx.org/Layouts/JASATeX +export/templates/Articles/Journal_of_the_Acoustical_Society_of_America.* + +# jss.cls +export/templates/Articles/Journal_of_Statistical_Software_.* + +# LilyPond-book version 2.13 or newer. +export/examples/(|ja/)Modules/LilyPond_Book_(dvi|pdf|lyx2[23]).* + +# maa-monthly.sty (not in TeXLive) +# http://www.maa.org/sites/default/files/pdf/pubs/maa-monthly.sty +export/templates/Articles/Mathematical_Monthly.* + +# knitr, RJournal.sty (not in TeXLive) +# incompatible with fontspec? +export/templates/Articles/R_Journal_.* + +# R (http://www.r-project.org) +# R package knitr (http://yihui.name/knitr) +export/examples/(|ja/)Modules/Sweave_(dvi|pdf|lyx2[23]).* +export/examples/(|ja/)Modules/Rnw.*knitr.*_(dvi|pdf|lyx2[23]).* + +# obsolete versions of documentclass or style files +export/templates/Obsolete/.* + +# svjour3.cls, svglov2.clo +# see https://wiki.lyx.org/Examples/Springer +export/templates/Articles/Springers_Global_Journal_Template.* + +# svmono.cls +export/templates/examples/Books/Springer_Monographs.* +# svmult.cls +export/templates/examples/Books/Springer_Contributed_Books.* + + +# TeXLive does not set up Hebrew 8-bit TeX fonts +# We use the external package "culmus-latex" +# https://sourceforge.net/projects/ivritex/files/culmus-latex/culmus-latex-0.7/ +# With current "culmus-latex", dvipdfm (.*_pdf3) fails because some of the +# fonts are type1 "*.pfa" while dvipdfm only works with binary "*.pfb" fonts: +# dvipdfmx:warning: Could not locate a virtual/physical font for TFM "david". +# dvipdfmx:warning: >> This font is mapped to a physical font "DavidCLM-Medium.pfa". +# dvipdfmx:warning: >> Please check if kpathsea library can find this font: DavidCLM-Medium.pfa +# but: `kpsewhich DavidCLM-Medium.pfa` finds the font! +# Converting with `t1binary` (from t1utils) helps. +export/.*/he/.*((dvi|pdf.?)|_texF) +export/export/latex/unicodesymbols/.*_he.*pdf2 +export/export/latex/(supported-languages|he-scrarticle).* +export/export/latex/languages/supported-languages_.*_pdf3 + + +# Japanese (platex) +# dvips requires special font setup for kanji. +# Debian's texlive-lang-japanese does this automatically, tlmgr not. +# Without proper font setup setup, +# * the Postscript file has wrong and missing characters +# (without error or warning), +# * as a consequence, `ps2pdf` fails for some documents: +export/.*/ja/.*_pdf -export/templates/IUCr-article_(dvi|pdf).* -export/templates/ja_beamer-conference-ornate-20min_(pdf2|pdf[45]_(texF|systemF)) -export/templates/es_beamer-conference-ornate-20min_(pdf2|pdf[45]_(texF|systemF)) -export/templates/kluwer_.*_systemF -export/examples/modernCV_pdf4_(tex|system)F -export/templates/ectaart_(dvi3|pdf5)_(tex|system)F +# "exotic" languages require "exotic" *system* fonts, e.g. +# Monomakh Unicode (from Debian package "texlive-fonts-extra" or sci.ponomar.net/fonts.html) +# Noto Serif Devanagari (from Debian package "fonts-noto-hinted") +export/export/latex/languages/supported-languages_polyglossia_.*_systemF -# 1.) missing farsi package with lfeenc.def -# 2.) LuaTeX does not support Farsi yet. See: -# https://github.com/reutenauer/polyglossia/commit/ccb0e9e2c6411170ad779b05ff5076f1193cc323 -export/examples/fa/splash_(dvi|pdf|pdf[23]|(dvi3|pdf4|pdf5)_(texF|systemF)) +Sublabel: needauth +################## +# Tests that require authorisation to call external programs. +export/examples/External_Material/Gnuplot.* +export/examples/Graphics_and_Insets/Minted_File_Listing.* +export/examples/Graphics_and_Insets/Minted_Listings.* Sublabel: erratic -# Tests depending on local configuration, OS, TeX distribution, -# package versions, or the phase of the moon. -# TODO: use *erratic* only for the phase-of-moon dependency. +################# +# Tests depending on local configuration, or the phase of the moon. -# for unknonwn reasons, seminar fails at some places: -export/examples/(|fr/)seminar_.* -# Export fails the first time but works the second time. (Why?) -export/doc/es/Customization_pdf4_texF +Sublabel: varying_versions +########################## +# If a test with this tag fails, this may be due to, e.g., the OS +# or the version of a dependency. +# Encoding cp858 only supported by some iconv variants +# Gnu iconv only supports it, if configured with "--enable-extra-encodings" +# (see https://www.gnu.org/software/libiconv/) +export/export/latex/inputenc-cp858_pdf2 -Sublabel: 1wrong 2output +# Fails with TL18 from Debian/stable (resolved with TexLive 19) +# ------------------------------------------------------------- + +# since TL18, Babel-Ukrainean uses UTF-8 if it detects Xe/LuaTeX (for "\date"). +# * garbled output with LuaTeX + TeX fonts (since TL18) +# * missing chars with XeTeX + TeX fonts (see invertedTests) +export/(doc|examples)/uk/(Intro|Welcome)_.*_texF + +# Polyglossia bug with LuaTeX and LTR languages (Arab, Hebrew and Farsi): +# Undefined control sequences "\luatexpardir" and "\luatextextdir". +# https://github.com/reutenauer/polyglossia/commit/ccb0e9e2c6411170ad779b05ff5076f1193cc323 +export/.*/ar/.*(dvi3|pdf5)_systemF +export/.*/fa/Welcome_(dvi3|pdf5)_systemF + +# Beamer fails also with XeTeX +export/examples/ar/Presentations/Beamer_.*_systemF + +# terminal message: "Language hebrew not found in language.dat.lua" +export/.*/he/.*(dvi3|pdf5)_systemF +export/export/latex/languages/supported-languages_(dvi3|pdf5)_systemF + +Fails with TL19 but solved in TL20 + +# Babel with non-TeX fonts fails for some supported languages +# Lithuanian: L7x font encoding also set with LuaTeX and non-TeX fonts +# (error but output OK) +export/export/latex/languages/supported-languages_babel_non-tex_(dvi3|pdf5)_systemF + +# Babel + XeTeX and system fonts fails due to a babel bug: +# https://www.mail-archive.com/search?l=mid&q=20190831012945.zl76dbstyijfqgdh%40boogie +# The error is the following, although remoing the referenced language just leads +# to a different error: +# +# ! File ended while scanning use of \bbl@inireader. +# +# \par +# l.3 \input{babel-german-switzerland} +# I suspect you have forgotten a `}', causing me +export/export/latex/languages/supported-languages_babel_non-tex_pdf4_systemF + + +Sublabel: wrong_output +###################### # Export does not fail but the resulting document has errors. # # These tests are actually not "unreliable" (different result for different # runs) but *invalid* (not measuring what they should measure). -# seminar.sty uses Postscript specials -# -> wrong output with pdflatex/LuaTeX and DVI (missing landscape slides). -export/examples/(|fr/)seminar_pdf[25].* -export/examples/(|fr/)seminar_dvi.* - -# Babel-Spanish uses Babel's "strings" feature to define -# separate auto-strings using UTF-8 literals. -# Babel uses the "unicode" strings if it detects XeTeX or LuaTeX. -# This is wrong for Xe/Lua with 8-bit TeX-fonts. -# set inputenc to utf8? -# (Changing the default in lib/languages requires more tests for utf8 first.) -export/examples/es/linguistics_pdf4_texF -export/doc/es/.*_(pdf[45]|dvi3)_texF - -# Non-ASCII char in verbatim environment is dropped (with a warning when -# doing this interactively) with ASCII (and hence also with XeTeX). -export/doc/(|de/|es/|fr/)EmbeddedObjects_pdf4_texF +# seminar.sty uses Postscript specials or PGF resulting in problems with slide +# size and orientation depending on the output format and viewing application. +# For details see Troubleshooting and the Appendix in examples/seminar.lyx. +export/examples/(|fr/)Presentations/Seminar_(dvi|pdf).* + +# Babel-Spanish uses UTF-8 encoded strings if it detects Xe/LuaTeX. +# With "language default" (=="latin9"), Babel strings come out wrong +# when compiling with Xe/LuaTeX and TeX-fonts. +export/export/latex/es_auto-legacy_texF + +# since TL18, Babel-Ukrainean uses UTF-8 if it detects Xe/LuaTeX (for "\date"). +# * garbled output with LuaTeX + TeX fonts (since TL18) +# (date and chapter headings) +# * missing chars with XeTeX + TeX fonts (see invertedTests) +export/export/latex/languages/uk-babel-date-missing-chars-with-ascii_.*_texF +export/doc/uk/Intro_.*_texF + +# Although DejaVu fonts are set and output is correct for pdf5, +# for dvi3 the Russian characters are not shown (the problem currently +# is obvious to see). +export/examples/ru/Graphics_and_Insets/XY-(Pic|Figure)_dvi3_systemF + + +# LyX issues +# ********** + +#6197: „amssymb” with Polish babel: name clash \lll ("<<<" vs. "ł") +# Wrong output with pdflatex +# missing character with XeTeX/LuaTeX and Babel (-> invertedTests) +export/export/latex/6197-polish-amssymb_(dvi|pdf).? + +# The following document mixes inTitle layouts and leads to +# likely unintended output. See #10347 +export/export/mixing_inTitle_layouts_(pdf|dvi).* + +# Characters with accents are not output +# Reported to LuaTeX mailing list here: +# https://www.mail-archive.com/search?l=mid&q=20160831134006.4fewxothddqfeyw4%40steph +# (not expecting much attention because DVI is not given high priority) +export/doc/es/Customization_dvi3_systemF +export/examples/(|es/)Curricula_Vitae/Europe_CV_dvi3_.*F + +# `acmart` requires PDFlatex: +# * Missing page number in DVI, Postscript, and PDF (ps2pdf) +# * "PDF (ps2pdf)" fails (problem with Postscript) in TeXLive < 2018 +# "Postscript" not tested by ctest autotests +export/templates/Articles/Association_for_Computing_Machinery_dvi.* +export/templates/Articles/Association_for_Computing_Machinery_pdf + +# lyx2lyx back-conversion of "Date" info-inset writes +# the name of the day in English instead of Japanese. +export/examples/ja/Multilingual_Typesetting_.*_lyx.* + +# No localization of auto-strings with Chinese language under 8-bit TeX +# (ctex font issue with texlive-lang-chinese (from Debian) under pdflatex) +# and only partial localization with non-tex-fonts: +export/export/latex/CJK/zh_CN-toc.* +export/doc/zh_CN/(Intro|Tutorial)_(dvi|pdf.?) + +# CJK languages don't change/reset the "language" (which is problematic for +# documents in a language that uses a non-standard font encoding) +# In a Greek document, Latin letters in a text part set to "Chinese" +# come out as Greek letters: +export/export/latex/CJK/el-zh_CN_utf8-cjk.* + +# Bug 3059: document language instead of "outer language" in table cells +# This document fails with "fontencoding=auto-legacy" +# and has wrong output (RTL in table cells) with all export formats. +export/export/latex/lyxbugs/3059.*