1 #LyX 2.3 created this file. For more info see http://www.lyx.org/
5 \save_transient_properties true
6 \origin /systemlyxdir/examples/Modules/
9 \usepackage[pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
10 \@savsf=1% This is to get around a hyperref+noweb interaction problem
14 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
16 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
17 \let\nwdocspar=\smallbreak
19 \use_default_options false
23 \maintain_unincluded_children false
25 \language_package default
28 \font_roman "times" "default"
29 \font_sans "helvet" "default"
30 \font_typewriter "courier" "default"
31 \font_math "auto" "auto"
32 \font_default_family default
33 \use_non_tex_fonts false
36 \font_sf_scale 100 100
37 \font_tt_scale 100 100
39 \use_dash_ligatures false
41 \default_output_format default
43 \bibtex_command default
44 \index_command default
45 \paperfontsize default
50 \use_package amsmath 1
51 \use_package amssymb 1
54 \use_package mathdots 1
55 \use_package mathtools 1
57 \use_package stackrel 1
58 \use_package stmaryrd 1
59 \use_package undertilde 1
61 \cite_engine_type default
65 \paperorientation portrait
76 \paragraph_separation indent
77 \paragraph_indentation default
79 \math_numbering_side default
84 \paperpagestyle default
85 \tracking_changes false
97 \begin_inset Newline newline
106 \begin_inset Newline newline
110 \begin_inset Flex URL
113 \begin_layout Plain Layout
115 mailto:kayvan@sylvan.com
127 \begin_layout Abstract
128 The listerrors program used to be compiled as a C program and installed
133 along with \SpecialChar LyX
134 in order to perform some simple re-formatting of noweb and
136 This document describes and implements the Python version of the same program.
139 \begin_layout Standard
140 \begin_inset CommandInset toc
141 LatexCommand tableofcontents
148 \begin_layout Section
152 \begin_layout Standard
153 The motivation for this program was \SpecialChar LyX
158 \begin_layout Plain Layout
160 \begin_inset Flex URL
163 \begin_layout Plain Layout
165 https://www.lyx.org/trac/ticket/190
176 \begin_inset Quotes eld
180 \begin_inset Quotes erd
186 \begin_layout Standard
188 \begin_inset Quotes eld
192 \begin_inset Quotes erd
195 ? Usually, \SpecialChar LyX
196 has great support for parsing of error messages.
197 For each error in the log file, \SpecialChar LyX
198 pops up an error box at that location
199 in the \SpecialChar LyX
201 The error scanning routines expect these errors to be in a certain format
202 (similar to \SpecialChar LaTeX
204 When dealing with Literate Programs, you have
205 \begin_inset Quotes eld
212 \begin_layout Plain Layout
214 \begin_inset Flex URL
217 \begin_layout Plain Layout
219 https://www.cs.tufts.edu/~nr/noweb/
224 for more information about noweb.
230 \begin_inset Quotes erd
233 as well as gcc error messages (and potentially others).
234 The listerrors program attempts to standardize these error messages to
235 a format that \SpecialChar LyX
236 can parse and react to.
239 \begin_layout Standard
240 In a nutshell, the problems with the old implementation of listerrors that
241 bug 190 refers to were::
244 \begin_layout Enumerate
245 It was a C program and it was installed in the user path in the same directory
248 Having such a generically named binary in, for example,
252 , was potentially confusing.
255 \begin_layout Enumerate
256 It required that noweb be installed on the compiling machine (the source
257 was extracted by noweb from
259 SRCDIR/examples/Literate.lyx
261 , compiled and installed by
262 \begin_inset Quotes eld
266 \begin_inset Quotes erd
272 \begin_layout Standard
273 The new version deals with these problems in the following fashion:
276 \begin_layout Enumerate
277 Both the example file (this document) and the program are to be added to
282 \begin_layout Enumerate
283 The program itself will be installed in
287 , along with other \SpecialChar LyX
288 -specific helper scripts.
291 \begin_layout Standard
292 In the design and implementation of this new
293 \begin_inset Quotes eld
297 \begin_inset Quotes erd
304 \begin_layout Plain Layout
305 See the Python home page (
306 \begin_inset Flex URL
309 \begin_layout Plain Layout
311 http://www.python.org
316 ) for more information.
321 language was chosen since it is fully multi-platform and provides a very
322 uniform and easy to read syntax.
323 This re-write also simplifies the code for
324 \begin_inset Quotes eld
328 \begin_inset Quotes erd
332 Python is installed by default on all modern Linux systems and is freely
333 available for all other platforms.
336 \begin_layout Standard
340 \begin_layout Plain Layout
345 \begin_layout Plain Layout
347 #!/usr/bin/python -tt
350 \begin_layout Plain Layout
352 """reformat noweb and compiler errors for LyX.
355 \begin_layout Plain Layout
359 \begin_layout Plain Layout
361 Expects to read from stdin and output to stdout.
364 \begin_layout Plain Layout
369 \begin_layout Plain Layout
373 \begin_layout Plain Layout
375 __author__ = "Kayvan A.
376 Sylvan <kayvan@sylvan.com>"
379 \begin_layout Plain Layout
381 __date__ = "$Date: 2005/07/18 09:42:26 $"
384 \begin_layout Plain Layout
386 __version__ = "$Revision: 1.5 $"
389 \begin_layout Plain Layout
391 __credits__ = """Edmar Wienskoski Jr.
392 <edmar-w-jr@technologist.com>
395 \begin_layout Plain Layout
397 original Literate support for LyX.
400 \begin_layout Plain Layout
402 Bernard Michael Hurley <berhardh@westherts.ac.uk>
405 \begin_layout Plain Layout
407 modifications to original listerrors."""
410 \begin_layout Plain Layout
412 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
415 \begin_layout Plain Layout
419 \begin_layout Plain Layout
424 \begin_layout Plain Layout
428 \begin_layout Plain Layout
433 \begin_layout Plain Layout
437 \begin_layout Plain Layout
439 if __name__ == "__main__":
442 \begin_layout Plain Layout
447 \begin_layout Plain Layout
457 \begin_layout Section
462 \begin_layout Standard
463 The following function mimics the \SpecialChar TeX
464 error message format.
467 \begin_layout Standard
471 \begin_layout Plain Layout
476 \begin_layout Plain Layout
478 def write_error(msg, tool = "noweb", line_number = 1):
481 \begin_layout Plain Layout
483 """Write out the given message in TeX error style.
486 \begin_layout Plain Layout
490 \begin_layout Plain Layout
494 \begin_layout Plain Layout
496 called like: write_error(msg, tool, line_number)."""
499 \begin_layout Plain Layout
501 print "! Build Error: ==> %s ==>
506 \begin_layout Plain Layout
517 \begin_layout Plain Layout
519 if type(msg) == type("str"): # simple string
522 \begin_layout Plain Layout
527 \begin_layout Plain Layout
529 else: # some kind of list (sequence or tuple)
532 \begin_layout Plain Layout
537 \begin_layout Plain Layout
542 \begin_layout Plain Layout
547 \begin_layout Plain Layout
551 \begin_layout Plain Layout
561 \begin_layout Section
565 \begin_layout Standard
566 The only complication in our filtering code is that some parsers might need
567 to push back lines that are read in to be read again later.
568 We solve this problem by implementing a
569 \begin_inset Quotes eld
573 \begin_inset Quotes erd
577 \begin_inset Quotes eld
581 \begin_inset Quotes erd
587 \begin_layout Standard
591 \begin_layout Plain Layout
596 \begin_layout Plain Layout
598 __lines = [] # lines pushed back
601 \begin_layout Plain Layout
605 \begin_layout Plain Layout
607 def getline(file = sys.stdin):
610 \begin_layout Plain Layout
612 """read a line from internal stack or from file.
615 \begin_layout Plain Layout
619 \begin_layout Plain Layout
621 optional file argument defaults to sys.stdin."""
624 \begin_layout Plain Layout
629 \begin_layout Plain Layout
634 \begin_layout Plain Layout
639 \begin_layout Plain Layout
644 \begin_layout Plain Layout
649 \begin_layout Plain Layout
651 line = file.readline()
654 \begin_layout Plain Layout
659 \begin_layout Plain Layout
663 \begin_layout Plain Layout
673 \begin_layout Standard
674 And now for the corresponding pushline function:
677 \begin_layout Standard
681 \begin_layout Plain Layout
686 \begin_layout Plain Layout
691 \begin_layout Plain Layout
693 "push a line onto the pushback stack."
696 \begin_layout Plain Layout
700 \begin_layout Plain Layout
705 \begin_layout Plain Layout
710 \begin_layout Plain Layout
715 \begin_layout Plain Layout
719 \begin_layout Plain Layout
729 \begin_layout Standard
730 The main() entry point function is extremely simple.
731 Note that this version of
732 \begin_inset Quotes eld
736 \begin_inset Quotes erd
739 takes no options and simply filters, attempting simply to match against
740 the known error message patterns.
741 The listerrors C program handled a single-character command-line argument
742 that the current code no longer needs.
746 \begin_layout Standard
750 \begin_layout Plain Layout
755 \begin_layout Plain Layout
760 \begin_layout Plain Layout
762 """Entry point for listerrors.
766 \begin_layout Plain Layout
770 \begin_layout Plain Layout
772 Reads stdin and writes to stdout.
776 \begin_layout Plain Layout
780 \begin_layout Plain Layout
785 \begin_layout Plain Layout
790 \begin_layout Plain Layout
795 \begin_layout Plain Layout
797 <<Check line against patterns and take action>>
800 \begin_layout Plain Layout
810 \begin_layout Standard
811 For each line read in, we need to find out if it matches any of our tools
812 (noweb, gcc, etc.) and act accordingly.
815 \begin_layout Standard
819 \begin_layout Plain Layout
821 <<Check line against patterns and take action>>=
824 \begin_layout Plain Layout
828 \begin_layout Plain Layout
830 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
833 \begin_layout Plain Layout
835 for predicate in try_patterns_dispatch:
838 \begin_layout Plain Layout
840 if predicate(line): break
843 \begin_layout Plain Layout
853 \begin_layout Section
854 Different Error Formats
857 \begin_layout Standard
858 The following sections handle the various error message formats that we
859 recognize in this program.
863 \begin_layout Subsection
867 \begin_layout Standard
868 Noweb errors are output on a single line, so examining just the current
872 \begin_layout Standard
876 \begin_layout Plain Layout
881 \begin_layout Plain Layout
886 \begin_layout Plain Layout
888 """see if line is a noweb error.
891 \begin_layout Plain Layout
895 \begin_layout Plain Layout
897 Returns 1 on success, 0 otherwise.
898 Outputs on stdout."""
901 \begin_layout Plain Layout
906 \begin_layout Plain Layout
908 <<Look for the unescaped angle-brackets in documentation>>
911 \begin_layout Plain Layout
913 <<Look for anything with double angle brackets>>
916 \begin_layout Plain Layout
918 <<Last ditch effort scan for specific strings>>
921 \begin_layout Plain Layout
926 \begin_layout Plain Layout
930 \begin_layout Plain Layout
940 \begin_layout Standard
941 First, we look for the
942 \begin_inset Quotes eld
945 unescaped < < in documentation chunk
946 \begin_inset Quotes erd
950 This is the only message with an associated line number from noweb.
953 \begin_layout Standard
957 \begin_layout Plain Layout
961 \begin_layout Plain Layout
963 <<Look for the unescaped angle-brackets in documentation>>=
966 \begin_layout Plain Layout
968 if string.find(line, ": unescaped << in documentation chunk") != -1:
971 \begin_layout Plain Layout
973 line_parts = string.split(line, ':')
976 \begin_layout Plain Layout
978 num_str = line_parts[1]
981 \begin_layout Plain Layout
983 num_len = len(num_str)
986 \begin_layout Plain Layout
991 \begin_layout Plain Layout
993 while i < num_len and (num_str[i] in string.digits): i = i + 1
996 \begin_layout Plain Layout
1001 \begin_layout Plain Layout
1003 write_error(":" + line_parts[2], "noweb", int(num_str))
1006 \begin_layout Plain Layout
1011 \begin_layout Plain Layout
1021 \begin_layout Standard
1022 Some noweb messages are simply about undefined chunks.
1023 These can be seen by looking for matching double-angle-brackets.
1026 \begin_layout Standard
1030 \begin_layout Plain Layout
1032 <<Look for anything with double angle brackets>>=
1035 \begin_layout Plain Layout
1039 \begin_layout Plain Layout
1044 \begin_layout Plain Layout
1046 left = string.find(line, "<<")
1049 \begin_layout Plain Layout
1051 if (left != -1) and ((left + 2) < len(line)) and
1056 \begin_layout Plain Layout
1058 (string.find(line[left+2:], ">>") != -1):
1061 \begin_layout Plain Layout
1063 write_error(line, "noweb");
1066 \begin_layout Plain Layout
1071 \begin_layout Plain Layout
1081 \begin_layout Standard
1082 Finally, here is an additional list of explicit strings to check for.
1085 \begin_layout Standard
1089 \begin_layout Plain Layout
1091 <<Last ditch effort scan for specific strings>>=
1094 \begin_layout Plain Layout
1099 \begin_layout Plain Layout
1101 msgs_to_try = ("couldn't open file",
1104 \begin_layout Plain Layout
1106 "couldn't open temporary file",
1109 \begin_layout Plain Layout
1111 "error writing temporary file",
1114 \begin_layout Plain Layout
1116 "ill-formed option",
1119 \begin_layout Plain Layout
1124 \begin_layout Plain Layout
1126 "Bad format sequence",
1129 \begin_layout Plain Layout
1131 "Can't open output file",
1134 \begin_layout Plain Layout
1136 "Can't open temporary file",
1139 \begin_layout Plain Layout
1141 "Capacity exceeded:",
1144 \begin_layout Plain Layout
1146 "Ignoring unknown option -",
1149 \begin_layout Plain Layout
1151 "This can't happen:",
1154 \begin_layout Plain Layout
1156 "non-numeric line number in")
1159 \begin_layout Plain Layout
1161 for msg in msgs_to_try:
1164 \begin_layout Plain Layout
1166 if string.find(line, msg) != -1:
1169 \begin_layout Plain Layout
1171 write_error(line, "noweb")
1174 \begin_layout Plain Layout
1179 \begin_layout Plain Layout
1184 \begin_layout Plain Layout
1194 \begin_layout Subsection
1198 \begin_layout Standard
1199 The gcc errors can be multi-line, with the following format:
1202 \begin_layout LyX-Code
1203 foo.c: In function `main':
1206 \begin_layout Standard
1210 \begin_layout Plain Layout
1212 foo.c:3: `bar' undeclared (first use in this function)
1215 \begin_layout Plain Layout
1217 foo.c:3: (Each undeclared identifier is reported only once
1220 \begin_layout Plain Layout
1224 \begin_layout Plain Layout
1226 foo.c:3: for each function it appears in.)
1229 \begin_layout Plain Layout
1231 foo.c:3: parse error before `x'
1239 \begin_layout Standard
1240 In order to parse this, the gcc error handler has to look ahead and return
1241 any and all lines that do not match the expected pattern.
1244 \begin_layout Standard
1248 \begin_layout Plain Layout
1250 <<Function Bodies>>=
1253 \begin_layout Plain Layout
1258 \begin_layout Plain Layout
1260 """See if line is a gcc error.
1261 Read ahead to handle all the lines.
1264 \begin_layout Plain Layout
1268 \begin_layout Plain Layout
1270 Returns 1 on success, 0 otherwise.
1271 Outputs on stdout."""
1274 \begin_layout Plain Layout
1279 \begin_layout Plain Layout
1281 <<Handle the gcc error message>>
1284 \begin_layout Plain Layout
1289 \begin_layout Plain Layout
1293 \begin_layout Plain Layout
1303 \begin_layout Standard
1304 The error message starts with a gcc header (as above) without an associated
1308 \begin_layout Standard
1312 \begin_layout Plain Layout
1314 <<Handle the gcc error message>>=
1317 \begin_layout Plain Layout
1319 first_space = string.find(line, ' ')
1322 \begin_layout Plain Layout
1324 if first_space > 1: # The smallest would be "X: "
1327 \begin_layout Plain Layout
1329 if line[first_space - 1] == ':':
1332 \begin_layout Plain Layout
1334 header_to_see = line[:first_space - 1]
1337 \begin_layout Plain Layout
1339 next_line = getline()
1342 \begin_layout Plain Layout
1344 if next_line and next_line[:first_space - 1] == header_to_see:
1347 \begin_layout Plain Layout
1349 num_end = first_space
1352 \begin_layout Plain Layout
1354 while next_line[num_end] in string.digits: num_end = num_end + 1
1357 \begin_layout Plain Layout
1359 if num_end > first_space: # good!
1362 \begin_layout Plain Layout
1364 <<Accumulate gcc error lines and print it>>
1367 \begin_layout Plain Layout
1369 else: # oops! Not a gcc error.
1372 \begin_layout Plain Layout
1377 \begin_layout Plain Layout
1382 \begin_layout Plain Layout
1384 pushline(next_line) # return this line to input stream
1387 \begin_layout Plain Layout
1397 \begin_layout Standard
1398 At the point in the code that we know that we are in the middle of an error
1399 message, we do the following:
1402 \begin_layout Standard
1406 \begin_layout Plain Layout
1408 <<Accumulate gcc error lines and print it>>=
1411 \begin_layout Plain Layout
1413 num_str = next_line[first_space:num_end]
1416 \begin_layout Plain Layout
1418 msgs = [line[first_space:]]
1421 \begin_layout Plain Layout
1423 msgs.append(next_line[num_end + 1:])
1426 \begin_layout Plain Layout
1428 header_to_see = next_line[:num_end]
1431 \begin_layout Plain Layout
1433 next_line = getline()
1436 \begin_layout Plain Layout
1438 while next_line and next_line[:num_end] == header_to_see:
1441 \begin_layout Plain Layout
1443 msgs.append(next_line[num_end + 1:])
1446 \begin_layout Plain Layout
1448 next_line = getline()
1451 \begin_layout Plain Layout
1453 if next_line: pushline(next_line)
1456 \begin_layout Plain Layout
1458 write_error(msgs, "gcc", int(num_str))
1461 \begin_layout Plain Layout
1466 \begin_layout Plain Layout
1476 \begin_layout Subsection
1480 \begin_layout Standard
1481 A xlc error message is easy to identify.
1482 Every error message starts with a quoted string with no spaces, a comma,
1484 \begin_inset Quotes eld
1488 \begin_inset Quotes erd
1491 , a space, and some variable text.
1492 The following routine tests if a given buffer line matches this criteria
1493 (this code would probably be simpler if I used the
1494 \begin_inset Quotes eld
1498 \begin_inset Quotes erd
1501 regexp module, but we don't really need the full regular expression engine
1506 \begin_layout Standard
1510 \begin_layout Plain Layout
1514 \begin_layout Plain Layout
1518 \begin_layout Plain Layout
1520 <<Function Bodies>>=
1523 \begin_layout Plain Layout
1528 \begin_layout Plain Layout
1530 """see if line is an xlc error.
1533 \begin_layout Plain Layout
1537 \begin_layout Plain Layout
1539 Returns 1 on success, 0 otherwise.
1540 Outputs on stdout."""
1543 \begin_layout Plain Layout
1548 \begin_layout Plain Layout
1550 if line[0] == '"': # This is the first character of all xlc errors
1553 \begin_layout Plain Layout
1555 next_quote = string.find(line, '"', 1)
1558 \begin_layout Plain Layout
1560 first_space = string.find(line, ' ')
1563 \begin_layout Plain Layout
1565 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1569 \begin_layout Plain Layout
1571 if line[first_space - 1:first_space + 6] == ", line ":
1574 \begin_layout Plain Layout
1576 num_start = num_end = first_space + 6
1579 \begin_layout Plain Layout
1581 while line[num_end] in string.digits: num_end = num_end + 1
1584 \begin_layout Plain Layout
1586 if num_end > num_start:
1589 \begin_layout Plain Layout
1591 write_error(line, "xlc", int(line[num_start : num_end]))
1594 \begin_layout Plain Layout
1599 \begin_layout Plain Layout
1604 \begin_layout Plain Layout
1609 \begin_layout Plain Layout
1619 \begin_layout Section
1623 \begin_layout Standard
1624 This project can be tangled from \SpecialChar LyX
1626 \begin_inset Quotes eld
1630 \begin_inset Quotes erd
1633 convertor to call a generic script that always extracts a chunk named
1638 Here is an example of such a generic script:
1641 \begin_layout LyX-Code
1645 \begin_layout LyX-Code
1646 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 NOWEB_OUTPUT_DIR=$2 sh
1649 \begin_layout Standard
1650 This section defines our build-script, which extracts the code.
1653 \begin_layout Standard
1657 \begin_layout Plain Layout
1662 \begin_layout Plain Layout
1667 \begin_layout Plain Layout
1669 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1672 \begin_layout Plain Layout
1674 if [ -z "$NOWEB_OUTPUT_DIR" ]; then NOWEB_OUTPUT_DIR=.; fi
1677 \begin_layout Plain Layout
1679 notangle -Rlisterrors ${NOWEB_SOURCE} > ${NOWEB_OUTPUT_DIR}/listerrors
1682 \begin_layout Plain Layout
1684 chmod +x ${NOWEB_OUTPUT_DIR}/listerrors
1687 \begin_layout Plain Layout
1697 \begin_layout Section
1701 \begin_layout Standard
1702 This section provides cross-references into the rest of the program.
1705 \begin_layout Subsection
1709 \begin_layout Standard
1713 \begin_layout Plain Layout
1725 \begin_layout Subsection
1729 \begin_layout Standard
1733 \begin_layout Plain Layout