1 #LyX 2.4 created this file. For more info see https://www.lyx.org/
5 \save_transient_properties true
6 \origin /systemlyxdir/examples/Modules/
9 \usepackage[pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
10 \@savsf=1% This is to get around a hyperref+noweb interaction problem
14 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
16 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
17 \let\nwdocspar=\smallbreak
19 \use_default_options false
23 \maintain_unincluded_children no
25 \language_package default
28 \font_roman "times" "default"
29 \font_sans "helvet" "default"
30 \font_typewriter "courier" "default"
31 \font_math "auto" "auto"
32 \font_default_family default
33 \use_non_tex_fonts false
37 \font_typewriter_osf false
38 \font_sf_scale 100 100
39 \font_tt_scale 100 100
41 \use_dash_ligatures false
43 \default_output_format default
45 \bibtex_command default
46 \index_command default
47 \paperfontsize default
52 \use_package amsmath 1
53 \use_package amssymb 1
56 \use_package mathdots 1
57 \use_package mathtools 1
59 \use_package stackrel 1
60 \use_package stmaryrd 1
61 \use_package undertilde 1
63 \cite_engine_type default
67 \paperorientation portrait
79 \paragraph_separation indent
80 \paragraph_indentation default
82 \math_numbering_side default
87 \paperpagestyle default
89 \tracking_changes false
92 \postpone_fragile_content false
96 \docbook_table_output 0
97 \docbook_mathml_prefix 1
105 \begin_inset Newline newline
114 \begin_inset Newline newline
118 \begin_inset Flex URL
121 \begin_layout Plain Layout
123 mailto:kayvan@sylvan.com
135 \begin_layout Abstract
136 The listerrors program used to be compiled as a C program and installed as
140 along with \SpecialChar LyX
141 in order to perform some simple re-formatting of noweb and GCC error messages.
142 This document describes and implements the Python version of the same program.
145 \begin_layout Standard
146 \begin_inset CommandInset toc
147 LatexCommand tableofcontents
154 \begin_layout Section
158 \begin_layout Standard
159 The motivation for this program was \SpecialChar LyX
164 \begin_layout Plain Layout
166 \begin_inset Flex URL
169 \begin_layout Plain Layout
171 https://www.lyx.org/trac/ticket/190
182 \begin_inset Quotes eld
186 \begin_inset Quotes erd
192 \begin_layout Standard
194 \begin_inset Quotes eld
198 \begin_inset Quotes erd
204 has great support for parsing of error messages.
205 For each error in the log file,
207 pops up an error box at that location in the \SpecialChar LyX
209 The error scanning routines expect these errors to be in a certain format (similar to \SpecialChar LaTeX
211 When dealing with Literate Programs,
213 \begin_inset Quotes eld
220 \begin_layout Plain Layout
222 \begin_inset Flex URL
225 \begin_layout Plain Layout
227 https://www.cs.tufts.edu/~nr/noweb/
232 for more information about noweb.
238 \begin_inset Quotes erd
241 as well as gcc error messages (and potentially others).
242 The listerrors program attempts to standardize these error messages to a format that \SpecialChar LyX
243 can parse and react to.
246 \begin_layout Standard
248 the problems with the old implementation of listerrors that bug 190 refers to were::
251 \begin_layout Enumerate
252 It was a C program and it was installed in the user path in the same directory as \SpecialChar LyX
254 Having such a generically named binary in,
261 was potentially confusing.
264 \begin_layout Enumerate
265 It required that noweb be installed on the compiling machine (the source was extracted by noweb from
267 SRCDIR/examples/Literate.lyx
270 compiled and installed by
271 \begin_inset Quotes eld
275 \begin_inset Quotes erd
281 \begin_layout Standard
282 The new version deals with these problems in the following fashion:
285 \begin_layout Enumerate
286 Both the example file (this document) and the program are to be added to the \SpecialChar LyX
290 \begin_layout Enumerate
291 The program itself will be installed in
296 along with other \SpecialChar LyX
297 -specific helper scripts.
300 \begin_layout Standard
301 In the design and implementation of this new
302 \begin_inset Quotes eld
306 \begin_inset Quotes erd
314 \begin_layout Plain Layout
315 See the Python home page (
316 \begin_inset Flex URL
319 \begin_layout Plain Layout
321 http://www.python.org
326 ) for more information.
331 language was chosen since it is fully multi-platform and provides a very uniform and easy to read syntax.
332 This re-write also simplifies the code for
333 \begin_inset Quotes eld
337 \begin_inset Quotes erd
341 Python is installed by default on all modern Linux systems and is freely available for all other platforms.
344 \begin_layout Standard
348 \begin_layout Plain Layout
353 \begin_layout Plain Layout
355 #!/usr/bin/python -tt
358 \begin_layout Plain Layout
360 """reformat noweb and compiler errors for LyX.
363 \begin_layout Plain Layout
367 \begin_layout Plain Layout
369 Expects to read from stdin and output to stdout.
372 \begin_layout Plain Layout
377 \begin_layout Plain Layout
381 \begin_layout Plain Layout
383 __author__ = "Kayvan A.
384 Sylvan <kayvan@sylvan.com>"
387 \begin_layout Plain Layout
390 2005/07/18 09:42:26 $"
393 \begin_layout Plain Layout
395 __version__ = "$Revision:
399 \begin_layout Plain Layout
401 __credits__ = """Edmar Wienskoski Jr.
402 <edmar-w-jr@technologist.com>
405 \begin_layout Plain Layout
407 original Literate support for LyX.
410 \begin_layout Plain Layout
412 Bernard Michael Hurley <berhardh@westherts.ac.uk>
415 \begin_layout Plain Layout
417 modifications to original listerrors."""
420 \begin_layout Plain Layout
422 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
425 \begin_layout Plain Layout
429 \begin_layout Plain Layout
435 \begin_layout Plain Layout
439 \begin_layout Plain Layout
444 \begin_layout Plain Layout
448 \begin_layout Plain Layout
450 if __name__ == "__main__":
453 \begin_layout Plain Layout
458 \begin_layout Plain Layout
468 \begin_layout Section
473 \begin_layout Standard
474 The following function mimics the \SpecialChar TeX
475 error message format.
478 \begin_layout Standard
482 \begin_layout Plain Layout
487 \begin_layout Plain Layout
494 \begin_layout Plain Layout
496 """Write out the given message in TeX error style.
499 \begin_layout Plain Layout
503 \begin_layout Plain Layout
507 \begin_layout Plain Layout
515 \begin_layout Plain Layout
524 \begin_layout Plain Layout
535 \begin_layout Plain Layout
537 if type(msg) == type("str"):
541 \begin_layout Plain Layout
546 \begin_layout Plain Layout
549 # some kind of list (sequence or tuple)
552 \begin_layout Plain Layout
557 \begin_layout Plain Layout
563 \begin_layout Plain Layout
568 \begin_layout Plain Layout
572 \begin_layout Plain Layout
582 \begin_layout Section
586 \begin_layout Standard
587 The only complication in our filtering code is that some parsers might need to push back lines that are read in to be read again later.
588 We solve this problem by implementing a
589 \begin_inset Quotes eld
593 \begin_inset Quotes erd
597 \begin_inset Quotes eld
601 \begin_inset Quotes erd
607 \begin_layout Standard
611 \begin_layout Plain Layout
616 \begin_layout Plain Layout
618 __lines = [] # lines pushed back
621 \begin_layout Plain Layout
625 \begin_layout Plain Layout
627 def getline(file = sys.stdin):
630 \begin_layout Plain Layout
632 """read a line from internal stack or from file.
635 \begin_layout Plain Layout
639 \begin_layout Plain Layout
641 optional file argument defaults to sys.stdin."""
644 \begin_layout Plain Layout
649 \begin_layout Plain Layout
654 \begin_layout Plain Layout
659 \begin_layout Plain Layout
664 \begin_layout Plain Layout
669 \begin_layout Plain Layout
671 line = file.readline()
674 \begin_layout Plain Layout
679 \begin_layout Plain Layout
683 \begin_layout Plain Layout
693 \begin_layout Standard
694 And now for the corresponding pushline function:
697 \begin_layout Standard
701 \begin_layout Plain Layout
706 \begin_layout Plain Layout
711 \begin_layout Plain Layout
713 "push a line onto the pushback stack."
716 \begin_layout Plain Layout
720 \begin_layout Plain Layout
725 \begin_layout Plain Layout
730 \begin_layout Plain Layout
735 \begin_layout Plain Layout
739 \begin_layout Plain Layout
749 \begin_layout Standard
750 The main() entry point function is extremely simple.
751 Note that this version of
752 \begin_inset Quotes eld
756 \begin_inset Quotes erd
759 takes no options and simply filters,
760 attempting simply to match against the known error message patterns.
761 The listerrors C program handled a single-character command-line argument that the current code no longer needs.
765 \begin_layout Standard
769 \begin_layout Plain Layout
774 \begin_layout Plain Layout
779 \begin_layout Plain Layout
781 """Entry point for listerrors.
785 \begin_layout Plain Layout
789 \begin_layout Plain Layout
791 Reads stdin and writes to stdout.
795 \begin_layout Plain Layout
799 \begin_layout Plain Layout
804 \begin_layout Plain Layout
809 \begin_layout Plain Layout
815 \begin_layout Plain Layout
817 <<Check line against patterns and take action>>
820 \begin_layout Plain Layout
830 \begin_layout Standard
831 For each line read in,
832 we need to find out if it matches any of our tools (noweb,
834 etc.) and act accordingly.
837 \begin_layout Standard
841 \begin_layout Plain Layout
843 <<Check line against patterns and take action>>=
846 \begin_layout Plain Layout
850 \begin_layout Plain Layout
852 try_patterns_dispatch = [ noweb_try,
857 \begin_layout Plain Layout
859 for predicate in try_patterns_dispatch:
862 \begin_layout Plain Layout
868 \begin_layout Plain Layout
878 \begin_layout Section
879 Different Error Formats
882 \begin_layout Standard
883 The following sections handle the various error message formats that we recognize in this program.
887 \begin_layout Subsection
891 \begin_layout Standard
892 Noweb errors are output on a single line,
893 so examining just the current line is enough.
896 \begin_layout Standard
900 \begin_layout Plain Layout
905 \begin_layout Plain Layout
910 \begin_layout Plain Layout
912 """see if line is a noweb error.
915 \begin_layout Plain Layout
919 \begin_layout Plain Layout
921 Returns 1 on success,
923 Outputs on stdout."""
926 \begin_layout Plain Layout
931 \begin_layout Plain Layout
933 <<Look for the unescaped angle-brackets in documentation>>
936 \begin_layout Plain Layout
938 <<Look for anything with double angle brackets>>
941 \begin_layout Plain Layout
943 <<Last ditch effort scan for specific strings>>
946 \begin_layout Plain Layout
951 \begin_layout Plain Layout
955 \begin_layout Plain Layout
965 \begin_layout Standard
968 \begin_inset Quotes eld
971 unescaped < < in documentation chunk
972 \begin_inset Quotes erd
976 This is the only message with an associated line number from noweb.
979 \begin_layout Standard
983 \begin_layout Plain Layout
987 \begin_layout Plain Layout
989 <<Look for the unescaped angle-brackets in documentation>>=
992 \begin_layout Plain Layout
996 unescaped << in documentation chunk") != -1:
999 \begin_layout Plain Layout
1001 line_parts = string.split(line,
1005 \begin_layout Plain Layout
1007 num_str = line_parts[1]
1010 \begin_layout Plain Layout
1012 num_len = len(num_str)
1015 \begin_layout Plain Layout
1020 \begin_layout Plain Layout
1022 while i < num_len and (num_str[i] in string.digits):
1026 \begin_layout Plain Layout
1031 \begin_layout Plain Layout
1033 write_error(":" + line_parts[2],
1038 \begin_layout Plain Layout
1043 \begin_layout Plain Layout
1053 \begin_layout Standard
1054 Some noweb messages are simply about undefined chunks.
1055 These can be seen by looking for matching double-angle-brackets.
1058 \begin_layout Standard
1062 \begin_layout Plain Layout
1064 <<Look for anything with double angle brackets>>=
1067 \begin_layout Plain Layout
1071 \begin_layout Plain Layout
1076 \begin_layout Plain Layout
1078 left = string.find(line,
1082 \begin_layout Plain Layout
1084 if (left != -1) and ((left + 2) < len(line)) and
1089 \begin_layout Plain Layout
1091 (string.find(line[left+2:],
1095 \begin_layout Plain Layout
1101 \begin_layout Plain Layout
1106 \begin_layout Plain Layout
1116 \begin_layout Standard
1118 here is an additional list of explicit strings to check for.
1121 \begin_layout Standard
1125 \begin_layout Plain Layout
1127 <<Last ditch effort scan for specific strings>>=
1130 \begin_layout Plain Layout
1135 \begin_layout Plain Layout
1137 msgs_to_try = ("couldn't open file",
1140 \begin_layout Plain Layout
1142 "couldn't open temporary file",
1145 \begin_layout Plain Layout
1147 "error writing temporary file",
1150 \begin_layout Plain Layout
1152 "ill-formed option",
1155 \begin_layout Plain Layout
1160 \begin_layout Plain Layout
1162 "Bad format sequence",
1165 \begin_layout Plain Layout
1167 "Can't open output file",
1170 \begin_layout Plain Layout
1172 "Can't open temporary file",
1175 \begin_layout Plain Layout
1177 "Capacity exceeded:",
1180 \begin_layout Plain Layout
1182 "Ignoring unknown option -",
1185 \begin_layout Plain Layout
1187 "This can't happen:",
1190 \begin_layout Plain Layout
1192 "non-numeric line number in")
1195 \begin_layout Plain Layout
1197 for msg in msgs_to_try:
1200 \begin_layout Plain Layout
1202 if string.find(line,
1206 \begin_layout Plain Layout
1212 \begin_layout Plain Layout
1217 \begin_layout Plain Layout
1222 \begin_layout Plain Layout
1232 \begin_layout Subsection
1236 \begin_layout Standard
1237 The gcc errors can be multi-line,
1238 with the following format:
1241 \begin_layout LyX-Code
1247 \begin_layout Standard
1251 \begin_layout Plain Layout
1254 `bar' undeclared (first use in this function)
1257 \begin_layout Plain Layout
1260 (Each undeclared identifier is reported only once
1263 \begin_layout Plain Layout
1267 \begin_layout Plain Layout
1270 for each function it appears in.)
1273 \begin_layout Plain Layout
1276 parse error before `x'
1284 \begin_layout Standard
1285 In order to parse this,
1286 the gcc error handler has to look ahead and return any and all lines that do not match the expected pattern.
1289 \begin_layout Standard
1293 \begin_layout Plain Layout
1295 <<Function Bodies>>=
1298 \begin_layout Plain Layout
1303 \begin_layout Plain Layout
1305 """See if line is a gcc error.
1306 Read ahead to handle all the lines.
1309 \begin_layout Plain Layout
1313 \begin_layout Plain Layout
1315 Returns 1 on success,
1317 Outputs on stdout."""
1320 \begin_layout Plain Layout
1325 \begin_layout Plain Layout
1327 <<Handle the gcc error message>>
1330 \begin_layout Plain Layout
1335 \begin_layout Plain Layout
1339 \begin_layout Plain Layout
1349 \begin_layout Standard
1350 The error message starts with a gcc header (as above) without an associated line number.
1353 \begin_layout Standard
1357 \begin_layout Plain Layout
1359 <<Handle the gcc error message>>=
1362 \begin_layout Plain Layout
1364 first_space = string.find(line,
1368 \begin_layout Plain Layout
1371 # The smallest would be "X:
1375 \begin_layout Plain Layout
1377 if line[first_space - 1] == ':':
1380 \begin_layout Plain Layout
1382 header_to_see = line[:first_space - 1]
1385 \begin_layout Plain Layout
1387 next_line = getline()
1390 \begin_layout Plain Layout
1392 if next_line and next_line[:first_space - 1] == header_to_see:
1395 \begin_layout Plain Layout
1397 num_end = first_space
1400 \begin_layout Plain Layout
1402 while next_line[num_end] in string.digits:
1403 num_end = num_end + 1
1406 \begin_layout Plain Layout
1408 if num_end > first_space:
1412 \begin_layout Plain Layout
1414 <<Accumulate gcc error lines and print it>>
1417 \begin_layout Plain Layout
1424 \begin_layout Plain Layout
1429 \begin_layout Plain Layout
1434 \begin_layout Plain Layout
1436 pushline(next_line) # return this line to input stream
1439 \begin_layout Plain Layout
1449 \begin_layout Standard
1450 At the point in the code that we know that we are in the middle of an error message,
1451 we do the following:
1454 \begin_layout Standard
1458 \begin_layout Plain Layout
1460 <<Accumulate gcc error lines and print it>>=
1463 \begin_layout Plain Layout
1465 num_str = next_line[first_space:num_end]
1468 \begin_layout Plain Layout
1470 msgs = [line[first_space:]]
1473 \begin_layout Plain Layout
1475 msgs.append(next_line[num_end + 1:])
1478 \begin_layout Plain Layout
1480 header_to_see = next_line[:num_end]
1483 \begin_layout Plain Layout
1485 next_line = getline()
1488 \begin_layout Plain Layout
1490 while next_line and next_line[:num_end] == header_to_see:
1493 \begin_layout Plain Layout
1495 msgs.append(next_line[num_end + 1:])
1498 \begin_layout Plain Layout
1500 next_line = getline()
1503 \begin_layout Plain Layout
1509 \begin_layout Plain Layout
1516 \begin_layout Plain Layout
1521 \begin_layout Plain Layout
1531 \begin_layout Subsection
1535 \begin_layout Standard
1536 A xlc error message is easy to identify.
1537 Every error message starts with a quoted string with no spaces,
1541 \begin_inset Quotes eld
1545 \begin_inset Quotes erd
1550 and some variable text.
1551 The following routine tests if a given buffer line matches this criteria (this code would probably be simpler if I used the
1552 \begin_inset Quotes eld
1556 \begin_inset Quotes erd
1560 but we don't really need the full regular expression engine here).
1564 \begin_layout Standard
1568 \begin_layout Plain Layout
1572 \begin_layout Plain Layout
1576 \begin_layout Plain Layout
1578 <<Function Bodies>>=
1581 \begin_layout Plain Layout
1586 \begin_layout Plain Layout
1588 """see if line is an xlc error.
1591 \begin_layout Plain Layout
1595 \begin_layout Plain Layout
1597 Returns 1 on success,
1599 Outputs on stdout."""
1602 \begin_layout Plain Layout
1607 \begin_layout Plain Layout
1610 # This is the first character of all xlc errors
1613 \begin_layout Plain Layout
1615 next_quote = string.find(line,
1620 \begin_layout Plain Layout
1622 first_space = string.find(line,
1626 \begin_layout Plain Layout
1628 if (next_quote != -1) and (first_space > next_quote):
1629 # no space inisde quotes
1632 \begin_layout Plain Layout
1634 if line[first_space - 1:first_space + 6] == ",
1638 \begin_layout Plain Layout
1640 num_start = num_end = first_space + 6
1643 \begin_layout Plain Layout
1645 while line[num_end] in string.digits:
1646 num_end = num_end + 1
1649 \begin_layout Plain Layout
1651 if num_end > num_start:
1654 \begin_layout Plain Layout
1658 int(line[num_start :
1662 \begin_layout Plain Layout
1667 \begin_layout Plain Layout
1672 \begin_layout Plain Layout
1677 \begin_layout Plain Layout
1687 \begin_layout Section
1691 \begin_layout Standard
1692 This project can be tangled from \SpecialChar LyX
1694 \begin_inset Quotes eld
1698 \begin_inset Quotes erd
1701 convertor to call a generic script that always extracts a chunk named
1706 Here is an example of such a generic script:
1709 \begin_layout LyX-Code
1713 \begin_layout LyX-Code
1714 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 NOWEB_OUTPUT_DIR=$2 sh
1717 \begin_layout Standard
1718 This section defines our build-script,
1719 which extracts the code.
1722 \begin_layout Standard
1726 \begin_layout Plain Layout
1731 \begin_layout Plain Layout
1736 \begin_layout Plain Layout
1738 if [ -z "$NOWEB_SOURCE" ];
1739 then NOWEB_SOURCE=listerrors.nw;
1743 \begin_layout Plain Layout
1745 if [ -z "$NOWEB_OUTPUT_DIR" ];
1746 then NOWEB_OUTPUT_DIR=.;
1750 \begin_layout Plain Layout
1752 notangle -Rlisterrors ${NOWEB_SOURCE} > ${NOWEB_OUTPUT_DIR}/listerrors
1755 \begin_layout Plain Layout
1757 chmod +x ${NOWEB_OUTPUT_DIR}/listerrors
1760 \begin_layout Plain Layout
1770 \begin_layout Section
1774 \begin_layout Standard
1775 This section provides cross-references into the rest of the program.
1778 \begin_layout Subsection
1782 \begin_layout Standard
1786 \begin_layout Plain Layout
1798 \begin_layout Subsection
1802 \begin_layout Standard
1806 \begin_layout Plain Layout