Begin 2.4.1 development

[lyx.git] / UPGRADING

How do I upgrade my existing LyX system to version 2.4.x?
---------------------------------------------------------

* Upgrading from LyX 2.3.x

The format of preference and session files has changed. LyX 2.4.x is able to
read old files but will save them in the new format.

The format of layout files has changed but, as before, layout2layout.py will
convert older versions to the new format automatically.

* Upgrading from LyX 2.2.x:

The format of preference and session files has changed. LyX 2.3.x is able to
read old files but will save them in the new format.

The format of layout files has changed but, as before, layout2layout.py will
convert older versions to the new format automatically.

The external_templates file has been split into one file per template,
which are now located in lib/xtemplates/*.xtemplate. This makes it easier
to add new templates or modify existing ones. If you have modified
the external_templates file, you will have to move the modifications to
the respective *.xtemplate file manually.

If using TeX fonts and en- and em-dashes are output as font ligatures,
when exporting documents containing en- and em-dashes to the format of
LyX 2.0 or earlier, the following line has to be manually added to the
unicodesymbols file of that LyX version:<br>
0x200b "\\hspace{0pt}" "" "" "" "" # ZERO WIDTH SPACE<br>
This avoids "uncodable character" issues if the document is actually
loaded by that LyX version. LyX 2.1 and later versions already have the
necessary definition in their unicodesymbols file.

If trying to compile documents using R scripts and sweave/knitr, LyX
2.3.x would not allow for re-running the R scripts, unless the user:
1) explicitly disables the "Forbid use of needauth converters"
option in the LyX preferences;
2) provides explicit consent to the use of the converter on the first
compilation of the R-enhanced document.

* Upgrading from LyX 2.1.x:

The format of preference and session files has changed. LyX 2.2.x is able to
read old files but will save them in the new format.

The format of layout files has changed but, as before, layout2layout.py will
convert older versions to the new format automatically.

The prefix for subsections in labels and references has been changed from
"sub:" to "subsec:" in order to avoid a clash with subfloats (conflicting
\subref command, see bug #7550). Files are automatically converted to the new
scheme. Please assure that you adapt external refstyle or prettyref definitions
and your own layout files.

BibTeX errors are now processed and cause LyX to show the errors dialog.
Before, these errors were ignored, which means that it may happen that
documents that compiled without error with a previous version now
compile with error. However, because now in 2.2.x users can click on
the "Show Output Anyway" button, the document can still be viewed.

Missing characters in the output are now reported as errors. This leads
to error reports for documents that compiled without error before.
However, the error was always present but went undetected!

Documents using Times fonts and containing Greek characters may now fail
to compile under pdflatex for users of MikTeX due to an automatically
half-installed "grtimes" package. A workaround in LyX was removed as it
stands in the way of alternative approaches (see bug #6469).

With LuaTeX, LyX now uses polyglossia instead of babel if the language
package option "Automatic" is selected. In order to use babel, select
"Always babel" instead. This may be needed if a document uses code that
is specific to babel.

* Upgrading from LyX 2.0.x:

Python version >=2.4 is now required.

Python version >3.0 is still not yet supported.

* Upgrading from LyX 1.6.x:

The typeset of your documents with non-english language can slightly
change in case of math environments and floats. LyX 2.0.x now has its
own translation machinery for the strings that are not translated by
babel.

The format of preference and session files has changed. LyX 2.0.x is
able to read old files but will save them in the new format.

The format of layout files has changed but, as before, layout2layout.py
will convert older versions to the new format automatically.

The syntax of the languages file has been changed. If you use a
modified languages file, you will need to adapt it to the new syntax.

There has been a large change in how Flex insets are named.
When exporting back to 1.6.x format user-defined flex insets will not
be properly reverted. See RELEASE-NOTES for details.

The UI layout named "classic.ui" and some localized keyboard bindings
(sv, pt, fi) are not being shipped anymore.

* Upgrading from LyX 1.5.x:

The format of preference and session files has changed. LyX 1.6.x is
able to read old files but will save them in the new format.

The format of layout files has changed but, as before,
layout2layout.py will convert older versions to the new format
automatically.

* Upgrading from LyX 1.4.x:

The biggest change in 1.5 is the switch to Unicode. Please refer to
the section "Document transfer" below for some things you might take
into account before upgrading.

The format of the preferences file has changed slightly. LyX 1.5.x is
able to read old preferences files, but it will save them in the new
format, so it is not possible to run LyX 1.4.x and 1.5.x with the same
personal configuration directory. If you are upgrading from 1.4.x and
do not intend to continue using 1.4.x, you should delete your existing
preferences file and allow LyX to create a new one.

The list of recently open files is now stored in a different location.
It will therefore be reset when upgrading from LyX 1.4.x.

The format of the layout files has also changed, but LyX 1.5.x uses a
converter layout2layout.py written in python that will convert old layout
files on the fly (see below, section "Document transfer").

* Upgrading from LyX 1.3.x:

The format of the external template file has changed substantially with
LyX 1.4.0. Automatic conversion is not available, so you need to convert
your external templates manually. The new format of the external template
configuration file is described in chapter 6.5 of the Customization Guide.

* Upgrading from LyX 1.2.x:

Since 1.3.0, you have to do the following changes:

One of the perennial bug bears of LyX users in the past has been that
they have had to run Edit->Reconfigure when starting their new version
of the code for the first time. Strange and wonderful things would
often result if they forgot to do this, so LyX 1.3.0 now runs
Edit->Reconfigure automatically the first time the program is run.

If you have your own layout files, you may need to update them a little:

- floats are now defined in the layout file, using the "Float"..."End"
  construct. In most cases, adding "Input stdfloats.inc" to your layout
  file is enough.

- counters are also defined in the layout files, using the
  "Counter"..."End" construct. As for floats, adding "Input
  stdfloats.inc" is probably a good idea.

* Upgrading from LyX 1.1.x:

- all layout files should have a "DefaultStyle" entry

- the "Latex" font style does not exist anymore. If you really need
  its functionality, consider using the "PassThru" keyword instead.

The new layout format keywords are described in the Customization
manual.

If you have your own binding files (especially math.bind), you will
have to update them

- math-insert now takes a latex macro name as argument, so that
  "math-insert sqrt" should now be "\math-insert \sqrt"

- math-greek-toggle is now gone, and should be replaced by explicit
  bindings like

  \bind "M-m g a"      "math-insert \alpha"


Build requirements
------------------

LyX 2.0 uses the Qt 4.x toolkit (see INSTALL file).

If you do not have the JPEG library installed, you may need to install it
before you can use the graphics capabilities of LyX. If you do not have the
ImageMagick command-line tools installed, you will need to modify the default
set up of LyX, or install them, in order to get previews of your document's
graphics.

Document transfer
-----------------

* Compatibility with older documents/layouts

LyX 2.0.x uses an external python script, lyx2lyx, to import documents
written using previous versions of LyX. All versions of LyX as far back as
0.10 are supported.

Of course, this means that you must have python (>= 2.3.4, <3)
installed in order to use LyX 2.0.x with your old documents.

lyx2lyx also has the framework in place to be able to convert documents to an
earlier format (which requires python 2.3.4 at least). However, these
converters have only been written for the conversion from 2.0.x to 1.6.x,
1.5.x, 1.4.x and 1.3.x, so versions of LyX older than 1.3.0 will NOT be able to
read documents saved with LyX 2.0.x. The conversion from 2.0.x to 1.6.x-1.3.x
is lossless as long as no new features are used. lyx2lyx tries hard to find
something equivalent for new features such as boxes, but this is known to fail
sometimes. LyX 1.6.9 contains an updated lyx2lyx that can read documents in
2.0.x format.

Furthermore, LyX uses a converter layout2layout.py, also written in python
that will convert old layout files on the fly. You can also call it manually
on your layout files if you want to convert them to 2.0.x format permanently.

* Formatted references

Before version 2.0, LyX used the LaTeX package "prettyref" to produce 
"formatted references", such as "Section 2.1". This package has several 
shortcomings when used in a non-English environment, not least of which is 
that it has no mechanism for internationalization. (See bug #6421 and those
it references.) 

As of LyX 2.0, users can choose whether to use prettyref or, alternatively,
the "refstyle" package. The current version of refstyle, v0.5, ships with 
translations for several languages and provides an easy mechanism for users
to translate the references it produces into still other languages. It also 
defines many more commands than prettyref does, including, for example, ones
to produce "ranges", such as "Sections 2.1 to 2.3". Some LyX developers are 
already working with the refstyle maintainer, Danie Els, to make it work more 
easily  with LyX and to extend the translations it provides. (You are invited 
to contribute translations, too!)

Because many LyX users already have customized prettyref for their purposes,
LyX 1.6.x files opened in LyX 2.0 will continue to use prettref by default.
New LyX 2.0 files will use refstyle by default. Both can of course be changed 
in Document>Settings. Please be advised, however, that prettyref support is to
be considered deprecated: It may well be removed in LyX 2.1, and all users are
encouraged to adapt their layout files, etc, to refstyle.

Doing so is fairly simple. With prettyref, one has declarations such as:
    \newrefformat{for}(Formula \ref{#1}}
The refstyle equivalent is:
    \newref{for}{refcmd={Formula \ref{#1}}}
The translation is obviously trivial.

* Preparing for Unicode:

As of version 1.5.0, LyX uses Unicode internally. This is a major change that
affects documents and layouts likewise. We have tried to do out best to make the
transition as smooth as possible for you. However, there are some caveats:

- User layout files must be converted to UTF-8

  In versions prior to 1.5.0, layout styles were allowed to use non-ASCII names
  using the local encodings. LyX-1.5 and later assume that all layout files are
  UTF-8 encoded. This means that non-ASCII style names are still allowed
  but they must be valid UTF-8 strings. One way of doing the conversion
  is to use iconv. Using bash, the script below should work:

     #! /bin/sh

     cd /path/to/layouts
     for l in *
     do
       cp "$l" tmp.txt
       iconv -f latin1 -t utf8 tmp.txt -o "$l"
     done
     rm -f tmp.txt

- Inset encodings and Conversion from earlier LyX versions

  As part of the transition to unicode, lyx2lyx (the scripts used for
  converting back and forth between different versions of the lyx
  files) converts old .lyx files, which may use a number of different
  encodings, to UTF-8. This conversion depends on correctly
  identifying the language of the text. There were previously some
  edge-cases (insets embedded in different-language text type
  scenarios) in which the language was incorrectly identified, which
  caused some text to appear incorrectly after having upgraded from
  older versions. This has now been fixed. Unfortunately, however, the
  fix cannot be applied to files which have already been converted
  past format 249. So if you have already converted your old files
  (using a development version or release candidate), this fix won't
  help, unless you still have the originals lying around (and haven't
  yet made too many changes to the newer versions ;) ).

Generally, it is probably wise to keep a backup of the old version of your
files, at least until you are sure that the upgrade went smoothly (which it
almost always will).

* Languages/encodings and insets

One of the bugs fixed in LyX 1.5.0 is that previously, there were certain
specific cases in which the LaTeX generated did not correctly reflect
language/encoding transitions in and around insets (footnotes, LyX notes).
After much deliberation, it was decided not to change older files such that
they will still reflect the old LaTeX output; rather, they will now correctly
reflect the situation as it appears in the GUI. This means, however, that if
you mangled the text in the GUI in the older versions, in order that it
generate the correct LaTeX output, the LaTeX will now generate the mangled
text. If this is problematic for you, please get in touch with us on the
developers mailing list, we do have some possible solutions for this.

The effects of this will be more pronounced for RTL (Hebrew, Arabic, Farsi)
users -- though they affect users of other languages as well.

* Floatflt in 1.2.x and older

If you were previously (in LyX 1.1.x) using the floatflt paragraph
option to wrap text around a figure, it was necessary to modify this
for LyX 1.2.0 manually, as described in the manuals. The feature has
been re-implemented as "Floating figure" inset in 1.3.0. Old files will
be converted automatically, but you may want to convert the
1.2.x-style ERT constructs with the native solution (see section 3.8
of the Extended Features manual).

* Babel changes since 1.2.x

Since LyX 1.2.0, the babel package is loaded after the user-defined
preamble (because some packages really need to be loaded before
babel). If you relied, on babel being loaded before your own
definitions, you can add an extra "\usepackage{babel}" statement at
the beginning of your preamble.

http://www.lyx.org/trac/ticket/315