1 #LyX 2.3 created this file. For more info see http://www.lyx.org/
5 \save_transient_properties true
6 \origin /buildlyxdir/examples/Modules/
12 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
13 \@savsf=1% This is to get around a hyperref+noweb interaction problem
17 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
19 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
20 \let\nwdocspar=\smallbreak
22 \use_default_options false
26 \maintain_unincluded_children false
28 \language_package default
31 \font_roman "times" "default"
32 \font_sans "helvet" "default"
33 \font_typewriter "courier" "default"
34 \font_math "auto" "auto"
35 \font_default_family default
36 \use_non_tex_fonts false
39 \font_sf_scale 100 100
40 \font_tt_scale 100 100
42 \use_dash_ligatures false
44 \default_output_format pdf
46 \bibtex_command default
47 \index_command default
48 \paperfontsize default
53 \use_package amsmath 1
54 \use_package amssymb 1
57 \use_package mathdots 1
58 \use_package mathtools 1
60 \use_package stackrel 1
61 \use_package stmaryrd 1
62 \use_package undertilde 1
64 \cite_engine_type default
68 \paperorientation portrait
79 \paragraph_separation indent
80 \paragraph_indentation default
82 \math_numbering_side default
87 \paperpagestyle default
88 \tracking_changes false
100 \begin_inset Newline newline
109 \begin_inset Newline newline
113 \begin_inset Flex URL
116 \begin_layout Plain Layout
118 mailto:kayvan@sylvan.com
130 \begin_layout Abstract
131 The listerrors program used to be compiled as a C program and installed
136 along with \SpecialChar LyX
137 in order to perform some simple re-formatting of noweb and
139 This document describes and implements the Python version of the same program.
142 \begin_layout Standard
143 \begin_inset CommandInset toc
144 LatexCommand tableofcontents
151 \begin_layout Section
155 \begin_layout Standard
156 The motivation for this program was \SpecialChar LyX
161 \begin_layout Plain Layout
163 \begin_inset Flex URL
166 \begin_layout Plain Layout
168 https://www.lyx.org/trac/ticket/190
179 \begin_inset Quotes eld
183 \begin_inset Quotes erd
189 \begin_layout Standard
191 \begin_inset Quotes eld
195 \begin_inset Quotes erd
198 ? Usually, \SpecialChar LyX
199 has great support for parsing of error messages.
200 For each error in the log file, \SpecialChar LyX
201 pops up an error box at that location
202 in the \SpecialChar LyX
204 The error scanning routines expect these errors to be in a certain format
205 (similar to \SpecialChar LaTeX
207 When dealing with Literate Programs, you have
208 \begin_inset Quotes eld
215 \begin_layout Plain Layout
217 \begin_inset Flex URL
220 \begin_layout Plain Layout
222 https://www.cs.tufts.edu/~nr/noweb/
227 for more information about noweb.
233 \begin_inset Quotes erd
236 as well as gcc error messages (and potentially others).
237 The listerrors program attempts to standardize these error messages to
238 a format that \SpecialChar LyX
239 can parse and react to.
242 \begin_layout Standard
243 In a nutshell, the problems with the old implementation of listerrors that
244 bug 190 refers to were::
247 \begin_layout Enumerate
248 It was a C program and it was installed in the user path in the same directory
251 Having such a generically named binary in, for example,
255 , was potentially confusing.
258 \begin_layout Enumerate
259 It required that noweb be installed on the compiling machine (the source
260 was extracted by noweb from
262 SRCDIR/examples/Literate.lyx
264 , compiled and installed by
265 \begin_inset Quotes eld
269 \begin_inset Quotes erd
275 \begin_layout Standard
276 The new version deals with these problems in the following fashion:
279 \begin_layout Enumerate
280 Both the example file (this document) and the program are to be added to
285 \begin_layout Enumerate
286 The program itself will be installed in
290 , along with other \SpecialChar LyX
291 -specific helper scripts.
294 \begin_layout Standard
295 In the design and implementation of this new
296 \begin_inset Quotes eld
300 \begin_inset Quotes erd
307 \begin_layout Plain Layout
308 See the Python home page (
309 \begin_inset Flex URL
312 \begin_layout Plain Layout
314 http://www.python.org
319 ) for more information.
324 language was chosen since it is fully multi-platform and provides a very
325 uniform and easy to read syntax.
326 This re-write also simplifies the code for
327 \begin_inset Quotes eld
331 \begin_inset Quotes erd
335 Python is installed by default on all modern Linux systems and is freely
336 available for all other platforms.
339 \begin_layout Standard
343 \begin_layout Plain Layout
348 \begin_layout Plain Layout
350 #!/usr/bin/python -tt
353 \begin_layout Plain Layout
355 """reformat noweb and compiler errors for LyX.
358 \begin_layout Plain Layout
362 \begin_layout Plain Layout
364 Expects to read from stdin and output to stdout.
367 \begin_layout Plain Layout
372 \begin_layout Plain Layout
376 \begin_layout Plain Layout
378 __author__ = "Kayvan A.
379 Sylvan <kayvan@sylvan.com>"
382 \begin_layout Plain Layout
384 __date__ = "$Date: 2005/07/18 09:42:26 $"
387 \begin_layout Plain Layout
389 __version__ = "$Revision: 1.5 $"
392 \begin_layout Plain Layout
394 __credits__ = """Edmar Wienskoski Jr.
395 <edmar-w-jr@technologist.com>
398 \begin_layout Plain Layout
400 original Literate support for LyX.
403 \begin_layout Plain Layout
405 Bernard Michael Hurley <berhardh@westherts.ac.uk>
408 \begin_layout Plain Layout
410 modifications to original listerrors."""
413 \begin_layout Plain Layout
415 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
418 \begin_layout Plain Layout
422 \begin_layout Plain Layout
427 \begin_layout Plain Layout
431 \begin_layout Plain Layout
436 \begin_layout Plain Layout
440 \begin_layout Plain Layout
442 if __name__ == "__main__":
445 \begin_layout Plain Layout
450 \begin_layout Plain Layout
460 \begin_layout Section
465 \begin_layout Standard
466 The following function mimics the \SpecialChar TeX
467 error message format.
470 \begin_layout Standard
474 \begin_layout Plain Layout
479 \begin_layout Plain Layout
481 def write_error(msg, tool = "noweb", line_number = 1):
484 \begin_layout Plain Layout
486 """Write out the given message in TeX error style.
489 \begin_layout Plain Layout
493 \begin_layout Plain Layout
497 \begin_layout Plain Layout
499 called like: write_error(msg, tool, line_number)."""
502 \begin_layout Plain Layout
504 print "! Build Error: ==> %s ==>
509 \begin_layout Plain Layout
520 \begin_layout Plain Layout
522 if type(msg) == type("str"): # simple string
525 \begin_layout Plain Layout
530 \begin_layout Plain Layout
532 else: # some kind of list (sequence or tuple)
535 \begin_layout Plain Layout
540 \begin_layout Plain Layout
545 \begin_layout Plain Layout
550 \begin_layout Plain Layout
554 \begin_layout Plain Layout
564 \begin_layout Section
568 \begin_layout Standard
569 The only complication in our filtering code is that some parsers might need
570 to push back lines that are read in to be read again later.
571 We solve this problem by implementing a
572 \begin_inset Quotes eld
576 \begin_inset Quotes erd
580 \begin_inset Quotes eld
584 \begin_inset Quotes erd
590 \begin_layout Standard
594 \begin_layout Plain Layout
599 \begin_layout Plain Layout
601 __lines = [] # lines pushed back
604 \begin_layout Plain Layout
608 \begin_layout Plain Layout
610 def getline(file = sys.stdin):
613 \begin_layout Plain Layout
615 """read a line from internal stack or from file.
618 \begin_layout Plain Layout
622 \begin_layout Plain Layout
624 optional file argument defaults to sys.stdin."""
627 \begin_layout Plain Layout
632 \begin_layout Plain Layout
637 \begin_layout Plain Layout
642 \begin_layout Plain Layout
647 \begin_layout Plain Layout
652 \begin_layout Plain Layout
654 line = file.readline()
657 \begin_layout Plain Layout
662 \begin_layout Plain Layout
666 \begin_layout Plain Layout
676 \begin_layout Standard
677 And now for the corresponding pushline function:
680 \begin_layout Standard
684 \begin_layout Plain Layout
689 \begin_layout Plain Layout
694 \begin_layout Plain Layout
696 "push a line onto the pushback stack."
699 \begin_layout Plain Layout
703 \begin_layout Plain Layout
708 \begin_layout Plain Layout
713 \begin_layout Plain Layout
718 \begin_layout Plain Layout
722 \begin_layout Plain Layout
732 \begin_layout Standard
733 The main() entry point function is extremely simple.
734 Note that this version of
735 \begin_inset Quotes eld
739 \begin_inset Quotes erd
742 takes no options and simply filters, attempting simply to match against
743 the known error message patterns.
744 The listerrors C program handled a single-character command-line argument
745 that the current code no longer needs.
749 \begin_layout Standard
753 \begin_layout Plain Layout
758 \begin_layout Plain Layout
763 \begin_layout Plain Layout
765 """Entry point for listerrors.
769 \begin_layout Plain Layout
773 \begin_layout Plain Layout
775 Reads stdin and writes to stdout.
779 \begin_layout Plain Layout
783 \begin_layout Plain Layout
788 \begin_layout Plain Layout
793 \begin_layout Plain Layout
798 \begin_layout Plain Layout
800 <<Check line against patterns and take action>>
803 \begin_layout Plain Layout
813 \begin_layout Standard
814 For each line read in, we need to find out if it matches any of our tools
815 (noweb, gcc, etc.) and act accordingly.
818 \begin_layout Standard
822 \begin_layout Plain Layout
824 <<Check line against patterns and take action>>=
827 \begin_layout Plain Layout
831 \begin_layout Plain Layout
833 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
836 \begin_layout Plain Layout
838 for predicate in try_patterns_dispatch:
841 \begin_layout Plain Layout
843 if predicate(line): break
846 \begin_layout Plain Layout
856 \begin_layout Section
857 Different Error Formats
860 \begin_layout Standard
861 The following sections handle the various error message formats that we
862 recognize in this program.
866 \begin_layout Subsection
870 \begin_layout Standard
871 Noweb errors are output on a single line, so examining just the current
875 \begin_layout Standard
879 \begin_layout Plain Layout
884 \begin_layout Plain Layout
889 \begin_layout Plain Layout
891 """see if line is a noweb error.
894 \begin_layout Plain Layout
898 \begin_layout Plain Layout
900 Returns 1 on success, 0 otherwise.
901 Outputs on stdout."""
904 \begin_layout Plain Layout
909 \begin_layout Plain Layout
911 <<Look for the unescaped angle-brackets in documentation>>
914 \begin_layout Plain Layout
916 <<Look for anything with double angle brackets>>
919 \begin_layout Plain Layout
921 <<Last ditch effort scan for specific strings>>
924 \begin_layout Plain Layout
929 \begin_layout Plain Layout
933 \begin_layout Plain Layout
943 \begin_layout Standard
944 First, we look for the
945 \begin_inset Quotes eld
948 unescaped < < in documentation chunk
949 \begin_inset Quotes erd
953 This is the only message with an associated line number from noweb.
956 \begin_layout Standard
960 \begin_layout Plain Layout
964 \begin_layout Plain Layout
966 <<Look for the unescaped angle-brackets in documentation>>=
969 \begin_layout Plain Layout
971 if string.find(line, ": unescaped << in documentation chunk") != -1:
974 \begin_layout Plain Layout
976 line_parts = string.split(line, ':')
979 \begin_layout Plain Layout
981 num_str = line_parts[1]
984 \begin_layout Plain Layout
986 num_len = len(num_str)
989 \begin_layout Plain Layout
994 \begin_layout Plain Layout
996 while i < num_len and (num_str[i] in string.digits): i = i + 1
999 \begin_layout Plain Layout
1004 \begin_layout Plain Layout
1006 write_error(":" + line_parts[2], "noweb", int(num_str))
1009 \begin_layout Plain Layout
1014 \begin_layout Plain Layout
1024 \begin_layout Standard
1025 Some noweb messages are simply about undefined chunks.
1026 These can be seen by looking for matching double-angle-brackets.
1029 \begin_layout Standard
1033 \begin_layout Plain Layout
1035 <<Look for anything with double angle brackets>>=
1038 \begin_layout Plain Layout
1042 \begin_layout Plain Layout
1047 \begin_layout Plain Layout
1049 left = string.find(line, "<<")
1052 \begin_layout Plain Layout
1054 if (left != -1) and ((left + 2) < len(line)) and
1059 \begin_layout Plain Layout
1061 (string.find(line[left+2:], ">>") != -1):
1064 \begin_layout Plain Layout
1066 write_error(line, "noweb");
1069 \begin_layout Plain Layout
1074 \begin_layout Plain Layout
1084 \begin_layout Standard
1085 Finally, here is an additional list of explicit strings to check for.
1088 \begin_layout Standard
1092 \begin_layout Plain Layout
1094 <<Last ditch effort scan for specific strings>>=
1097 \begin_layout Plain Layout
1102 \begin_layout Plain Layout
1104 msgs_to_try = ("couldn't open file",
1107 \begin_layout Plain Layout
1109 "couldn't open temporary file",
1112 \begin_layout Plain Layout
1114 "error writing temporary file",
1117 \begin_layout Plain Layout
1119 "ill-formed option",
1122 \begin_layout Plain Layout
1127 \begin_layout Plain Layout
1129 "Bad format sequence",
1132 \begin_layout Plain Layout
1134 "Can't open output file",
1137 \begin_layout Plain Layout
1139 "Can't open temporary file",
1142 \begin_layout Plain Layout
1144 "Capacity exceeded:",
1147 \begin_layout Plain Layout
1149 "Ignoring unknown option -",
1152 \begin_layout Plain Layout
1154 "This can't happen:",
1157 \begin_layout Plain Layout
1159 "non-numeric line number in")
1162 \begin_layout Plain Layout
1164 for msg in msgs_to_try:
1167 \begin_layout Plain Layout
1169 if string.find(line, msg) != -1:
1172 \begin_layout Plain Layout
1174 write_error(line, "noweb")
1177 \begin_layout Plain Layout
1182 \begin_layout Plain Layout
1187 \begin_layout Plain Layout
1197 \begin_layout Subsection
1201 \begin_layout Standard
1202 The gcc errors can be multi-line, with the following format:
1205 \begin_layout LyX-Code
1206 foo.c: In function `main':
1209 \begin_layout Standard
1213 \begin_layout Plain Layout
1215 foo.c:3: `bar' undeclared (first use in this function)
1218 \begin_layout Plain Layout
1220 foo.c:3: (Each undeclared identifier is reported only once
1223 \begin_layout Plain Layout
1227 \begin_layout Plain Layout
1229 foo.c:3: for each function it appears in.)
1232 \begin_layout Plain Layout
1234 foo.c:3: parse error before `x'
1242 \begin_layout Standard
1243 In order to parse this, the gcc error handler has to look ahead and return
1244 any and all lines that do not match the expected pattern.
1247 \begin_layout Standard
1251 \begin_layout Plain Layout
1253 <<Function Bodies>>=
1256 \begin_layout Plain Layout
1261 \begin_layout Plain Layout
1263 """See if line is a gcc error.
1264 Read ahead to handle all the lines.
1267 \begin_layout Plain Layout
1271 \begin_layout Plain Layout
1273 Returns 1 on success, 0 otherwise.
1274 Outputs on stdout."""
1277 \begin_layout Plain Layout
1282 \begin_layout Plain Layout
1284 <<Handle the gcc error message>>
1287 \begin_layout Plain Layout
1292 \begin_layout Plain Layout
1296 \begin_layout Plain Layout
1306 \begin_layout Standard
1307 The error message starts with a gcc header (as above) without an associated
1311 \begin_layout Standard
1315 \begin_layout Plain Layout
1317 <<Handle the gcc error message>>=
1320 \begin_layout Plain Layout
1322 first_space = string.find(line, ' ')
1325 \begin_layout Plain Layout
1327 if first_space > 1: # The smallest would be "X: "
1330 \begin_layout Plain Layout
1332 if line[first_space - 1] == ':':
1335 \begin_layout Plain Layout
1337 header_to_see = line[:first_space - 1]
1340 \begin_layout Plain Layout
1342 next_line = getline()
1345 \begin_layout Plain Layout
1347 if next_line and next_line[:first_space - 1] == header_to_see:
1350 \begin_layout Plain Layout
1352 num_end = first_space
1355 \begin_layout Plain Layout
1357 while next_line[num_end] in string.digits: num_end = num_end + 1
1360 \begin_layout Plain Layout
1362 if num_end > first_space: # good!
1365 \begin_layout Plain Layout
1367 <<Accumulate gcc error lines and print it>>
1370 \begin_layout Plain Layout
1372 else: # oops! Not a gcc error.
1375 \begin_layout Plain Layout
1380 \begin_layout Plain Layout
1385 \begin_layout Plain Layout
1387 pushline(next_line) # return this line to input stream
1390 \begin_layout Plain Layout
1400 \begin_layout Standard
1401 At the point in the code that we know that we are in the middle of an error
1402 message, we do the following:
1405 \begin_layout Standard
1409 \begin_layout Plain Layout
1411 <<Accumulate gcc error lines and print it>>=
1414 \begin_layout Plain Layout
1416 num_str = next_line[first_space:num_end]
1419 \begin_layout Plain Layout
1421 msgs = [line[first_space:]]
1424 \begin_layout Plain Layout
1426 msgs.append(next_line[num_end + 1:])
1429 \begin_layout Plain Layout
1431 header_to_see = next_line[:num_end]
1434 \begin_layout Plain Layout
1436 next_line = getline()
1439 \begin_layout Plain Layout
1441 while next_line and next_line[:num_end] == header_to_see:
1444 \begin_layout Plain Layout
1446 msgs.append(next_line[num_end + 1:])
1449 \begin_layout Plain Layout
1451 next_line = getline()
1454 \begin_layout Plain Layout
1456 if next_line: pushline(next_line)
1459 \begin_layout Plain Layout
1461 write_error(msgs, "gcc", int(num_str))
1464 \begin_layout Plain Layout
1469 \begin_layout Plain Layout
1479 \begin_layout Subsection
1483 \begin_layout Standard
1484 A xlc error message is easy to identify.
1485 Every error message starts with a quoted string with no spaces, a comma,
1487 \begin_inset Quotes eld
1491 \begin_inset Quotes erd
1494 , a space, and some variable text.
1495 The following routine tests if a given buffer line matches this criteria
1496 (this code would probably be simpler if I used the
1497 \begin_inset Quotes eld
1501 \begin_inset Quotes erd
1504 regexp module, but we don't really need the full regular expression engine
1509 \begin_layout Standard
1513 \begin_layout Plain Layout
1517 \begin_layout Plain Layout
1521 \begin_layout Plain Layout
1523 <<Function Bodies>>=
1526 \begin_layout Plain Layout
1531 \begin_layout Plain Layout
1533 """see if line is an xlc error.
1536 \begin_layout Plain Layout
1540 \begin_layout Plain Layout
1542 Returns 1 on success, 0 otherwise.
1543 Outputs on stdout."""
1546 \begin_layout Plain Layout
1551 \begin_layout Plain Layout
1553 if line[0] == '"': # This is the first character of all xlc errors
1556 \begin_layout Plain Layout
1558 next_quote = string.find(line, '"', 1)
1561 \begin_layout Plain Layout
1563 first_space = string.find(line, ' ')
1566 \begin_layout Plain Layout
1568 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1572 \begin_layout Plain Layout
1574 if line[first_space - 1:first_space + 6] == ", line ":
1577 \begin_layout Plain Layout
1579 num_start = num_end = first_space + 6
1582 \begin_layout Plain Layout
1584 while line[num_end] in string.digits: num_end = num_end + 1
1587 \begin_layout Plain Layout
1589 if num_end > num_start:
1592 \begin_layout Plain Layout
1594 write_error(line, "xlc", int(line[num_start : num_end]))
1597 \begin_layout Plain Layout
1602 \begin_layout Plain Layout
1607 \begin_layout Plain Layout
1612 \begin_layout Plain Layout
1622 \begin_layout Section
1626 \begin_layout Standard
1627 This project can be tangled from \SpecialChar LyX
1629 \begin_inset Quotes eld
1633 \begin_inset Quotes erd
1636 convertor to call a generic script that always extracts a chunk named
1641 Here is an example of such a generic script:
1644 \begin_layout LyX-Code
1648 \begin_layout LyX-Code
1649 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 NOWEB_OUTPUT_DIR=$2 sh
1652 \begin_layout Standard
1653 This section defines our build-script, which extracts the code.
1656 \begin_layout Standard
1660 \begin_layout Plain Layout
1665 \begin_layout Plain Layout
1670 \begin_layout Plain Layout
1672 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1675 \begin_layout Plain Layout
1677 if [ -z "$NOWEB_OUTPUT_DIR" ]; then NOWEB_OUTPUT_DIR=.; fi
1680 \begin_layout Plain Layout
1682 notangle -Rlisterrors ${NOWEB_SOURCE} > ${NOWEB_OUTPUT_DIR}/listerrors
1685 \begin_layout Plain Layout
1687 chmod +x ${NOWEB_OUTPUT_DIR}/listerrors
1690 \begin_layout Plain Layout
1700 \begin_layout Section
1704 \begin_layout Standard
1705 This section provides cross-references into the rest of the program.
1708 \begin_layout Subsection
1712 \begin_layout Standard
1716 \begin_layout Plain Layout
1728 \begin_layout Subsection
1732 \begin_layout Standard
1736 \begin_layout Plain Layout