1 #LyX 2.3 created this file. For more info see http://www.lyx.org/
5 \save_transient_properties true
6 \origin /systemlyxdir/examples/
12 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
13 \@savsf=1% This is to get around a hyperref+noweb interaction problem
17 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
19 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
20 \let\nwdocspar=\smallbreak
22 \use_default_options false
26 \maintain_unincluded_children false
28 \language_package default
31 \font_roman "times" "default"
32 \font_sans "helvet" "default"
33 \font_typewriter "courier" "default"
34 \font_math "auto" "auto"
35 \font_default_family default
36 \use_non_tex_fonts false
39 \font_sf_scale 100 100
40 \font_tt_scale 100 100
42 \use_dash_ligatures false
44 \default_output_format pdf
46 \bibtex_command default
47 \index_command default
48 \paperfontsize default
53 \use_package amsmath 0
54 \use_package amssymb 0
57 \use_package mathdots 1
58 \use_package mathtools 0
60 \use_package stackrel 0
61 \use_package stmaryrd 0
62 \use_package undertilde 0
64 \cite_engine_type default
68 \paperorientation portrait
78 \paragraph_separation indent
79 \paragraph_indentation default
81 \math_numbering_side default
85 \paperpagestyle default
86 \tracking_changes false
99 \begin_inset Newline newline
108 \begin_inset Newline newline
112 \begin_inset Flex URL
115 \begin_layout Plain Layout
117 mailto:kayvan@sylvan.com
129 \begin_layout Abstract
130 The listerrors program used to be compiled as a C program and installed
135 along with \SpecialChar LyX
136 in order to perform some simple re-formatting of noweb and
138 This document describes and implements the Python version of the same program.
141 \begin_layout Standard
142 \begin_inset CommandInset toc
143 LatexCommand tableofcontents
150 \begin_layout Section
154 \begin_layout Standard
155 The motivation for this program was \SpecialChar LyX
160 \begin_layout Plain Layout
162 \begin_inset Flex URL
165 \begin_layout Plain Layout
167 http://www.lyx.org/trac/ticket/190
178 \begin_inset Quotes eld
182 \begin_inset Quotes erd
188 \begin_layout Standard
190 \begin_inset Quotes eld
194 \begin_inset Quotes erd
197 ? Usually, \SpecialChar LyX
198 has great support for parsing of error messages.
199 For each error in the log file, \SpecialChar LyX
200 pops up an error box at that location
201 in the \SpecialChar LyX
203 The error scanning routines expect these errors to be in a certain format
204 (similar to \SpecialChar LaTeX
206 When dealing with Literate Programs, you have
207 \begin_inset Quotes eld
214 \begin_layout Plain Layout
216 \begin_inset Flex URL
219 \begin_layout Plain Layout
221 http://www.eecs.harvard.edu/~nr/noweb
226 for more information about noweb.
232 \begin_inset Quotes erd
235 as well as gcc error messages (and potentially others).
236 The listerrors program attempts to standardize these error messages to
237 a format that \SpecialChar LyX
238 can parse and react to.
241 \begin_layout Standard
242 In a nutshell, the problems with the old implementation of listerrors that
243 bug 190 refers to were::
246 \begin_layout Enumerate
247 It was a C program and it was installed in the user path in the same directory
250 Having such a generically named binary in, for example,
254 , was potentially confusing.
257 \begin_layout Enumerate
258 It required that noweb be installed on the compiling machine (the source
259 was extracted by noweb from
261 SRCDIR/examples/Literate.lyx
263 , compiled and installed by
264 \begin_inset Quotes eld
268 \begin_inset Quotes erd
274 \begin_layout Standard
275 The new version deals with these problems in the following fashion:
278 \begin_layout Enumerate
279 Both the example file (this document) and the program are to be added to
284 \begin_layout Enumerate
285 The program itself will be installed in
289 , along with other \SpecialChar LyX
290 -specific helper scripts.
293 \begin_layout Standard
294 In the design and implementation of this new
295 \begin_inset Quotes eld
299 \begin_inset Quotes erd
306 \begin_layout Plain Layout
307 See the Python home page (
308 \begin_inset Flex URL
311 \begin_layout Plain Layout
313 http://www.python.org
318 ) for more information.
323 language was chosen since it is fully multi-platform and provides a very
324 uniform and easy to read syntax.
325 This re-write also simplifies the code for
326 \begin_inset Quotes eld
330 \begin_inset Quotes erd
334 Python is installed by default on all modern Linux systems and is freely
335 available for all other platforms.
338 \begin_layout Standard
342 \begin_layout Plain Layout
347 \begin_layout Plain Layout
349 #!/usr/bin/python -tt
352 \begin_layout Plain Layout
354 """reformat noweb and compiler errors for LyX.
357 \begin_layout Plain Layout
361 \begin_layout Plain Layout
363 Expects to read from stdin and output to stdout.
366 \begin_layout Plain Layout
371 \begin_layout Plain Layout
375 \begin_layout Plain Layout
377 __author__ = "Kayvan A.
378 Sylvan <kayvan@sylvan.com>"
381 \begin_layout Plain Layout
383 __date__ = "$Date: 2005/07/18 09:42:26 $"
386 \begin_layout Plain Layout
388 __version__ = "$Revision: 1.5 $"
391 \begin_layout Plain Layout
393 __credits__ = """Edmar Wienskoski Jr.
394 <edmar-w-jr@technologist.com>
397 \begin_layout Plain Layout
399 original Literate support for LyX.
402 \begin_layout Plain Layout
404 Bernard Michael Hurley <berhardh@westherts.ac.uk>
407 \begin_layout Plain Layout
409 modifications to original listerrors."""
412 \begin_layout Plain Layout
414 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
417 \begin_layout Plain Layout
421 \begin_layout Plain Layout
426 \begin_layout Plain Layout
430 \begin_layout Plain Layout
435 \begin_layout Plain Layout
439 \begin_layout Plain Layout
441 if __name__ == "__main__":
444 \begin_layout Plain Layout
449 \begin_layout Plain Layout
459 \begin_layout Section
464 \begin_layout Standard
465 The following function mimics the \SpecialChar TeX
466 error message format.
469 \begin_layout Standard
473 \begin_layout Plain Layout
478 \begin_layout Plain Layout
480 def write_error(msg, tool = "noweb", line_number = 1):
483 \begin_layout Plain Layout
485 """Write out the given message in TeX error style.
488 \begin_layout Plain Layout
492 \begin_layout Plain Layout
496 \begin_layout Plain Layout
498 called like: write_error(msg, tool, line_number)."""
501 \begin_layout Plain Layout
503 print "! Build Error: ==> %s ==>
508 \begin_layout Plain Layout
519 \begin_layout Plain Layout
521 if type(msg) == type("str"): # simple string
524 \begin_layout Plain Layout
529 \begin_layout Plain Layout
531 else: # some kind of list (sequence or tuple)
534 \begin_layout Plain Layout
539 \begin_layout Plain Layout
544 \begin_layout Plain Layout
549 \begin_layout Plain Layout
553 \begin_layout Plain Layout
563 \begin_layout Section
567 \begin_layout Standard
568 The only complication in our filtering code is that some parsers might need
569 to push back lines that are read in to be read again later.
570 We solve this problem by implementing a
571 \begin_inset Quotes eld
575 \begin_inset Quotes erd
579 \begin_inset Quotes eld
583 \begin_inset Quotes erd
589 \begin_layout Standard
593 \begin_layout Plain Layout
598 \begin_layout Plain Layout
600 __lines = [] # lines pushed back
603 \begin_layout Plain Layout
607 \begin_layout Plain Layout
609 def getline(file = sys.stdin):
612 \begin_layout Plain Layout
614 """read a line from internal stack or from file.
617 \begin_layout Plain Layout
621 \begin_layout Plain Layout
623 optional file argument defaults to sys.stdin."""
626 \begin_layout Plain Layout
631 \begin_layout Plain Layout
636 \begin_layout Plain Layout
641 \begin_layout Plain Layout
646 \begin_layout Plain Layout
651 \begin_layout Plain Layout
653 line = file.readline()
656 \begin_layout Plain Layout
661 \begin_layout Plain Layout
665 \begin_layout Plain Layout
675 \begin_layout Standard
676 And now for the corresponding pushline function:
679 \begin_layout Standard
683 \begin_layout Plain Layout
688 \begin_layout Plain Layout
693 \begin_layout Plain Layout
695 "push a line onto the pushback stack."
698 \begin_layout Plain Layout
702 \begin_layout Plain Layout
707 \begin_layout Plain Layout
712 \begin_layout Plain Layout
717 \begin_layout Plain Layout
721 \begin_layout Plain Layout
731 \begin_layout Standard
732 The main() entry point function is extremely simple.
733 Note that this version of
734 \begin_inset Quotes eld
738 \begin_inset Quotes erd
741 takes no options and simply filters, attempting simply to match against
742 the known error message patterns.
743 The listerrors C program handled a single-character command-line argument
744 that the current code no longer needs.
748 \begin_layout Standard
752 \begin_layout Plain Layout
757 \begin_layout Plain Layout
762 \begin_layout Plain Layout
764 """Entry point for listerrors.
768 \begin_layout Plain Layout
772 \begin_layout Plain Layout
774 Reads stdin and writes to stdout.
778 \begin_layout Plain Layout
782 \begin_layout Plain Layout
787 \begin_layout Plain Layout
792 \begin_layout Plain Layout
797 \begin_layout Plain Layout
799 <<Check line against patterns and take action>>
802 \begin_layout Plain Layout
812 \begin_layout Standard
813 For each line read in, we need to find out if it matches any of our tools
814 (noweb, gcc, etc.) and act accordingly.
817 \begin_layout Standard
821 \begin_layout Plain Layout
823 <<Check line against patterns and take action>>=
826 \begin_layout Plain Layout
830 \begin_layout Plain Layout
832 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
835 \begin_layout Plain Layout
837 for predicate in try_patterns_dispatch:
840 \begin_layout Plain Layout
842 if predicate(line): break
845 \begin_layout Plain Layout
855 \begin_layout Section
856 Different Error Formats
859 \begin_layout Standard
860 The following sections handle the various error message formats that we
861 recognize in this program.
865 \begin_layout Subsection
869 \begin_layout Standard
870 Noweb errors are output on a single line, so examining just the current
874 \begin_layout Standard
878 \begin_layout Plain Layout
883 \begin_layout Plain Layout
888 \begin_layout Plain Layout
890 """see if line is a noweb error.
893 \begin_layout Plain Layout
897 \begin_layout Plain Layout
899 Returns 1 on success, 0 otherwise.
900 Outputs on stdout."""
903 \begin_layout Plain Layout
908 \begin_layout Plain Layout
910 <<Look for the unescaped angle-brackets in documentation>>
913 \begin_layout Plain Layout
915 <<Look for anything with double angle brackets>>
918 \begin_layout Plain Layout
920 <<Last ditch effort scan for specific strings>>
923 \begin_layout Plain Layout
928 \begin_layout Plain Layout
932 \begin_layout Plain Layout
942 \begin_layout Standard
943 First, we look for the
944 \begin_inset Quotes eld
947 unescaped < < in documentation chunk
948 \begin_inset Quotes erd
952 This is the only message with an associated line number from noweb.
955 \begin_layout Standard
959 \begin_layout Plain Layout
963 \begin_layout Plain Layout
965 <<Look for the unescaped angle-brackets in documentation>>=
968 \begin_layout Plain Layout
970 if string.find(line, ": unescaped << in documentation chunk") != -1:
973 \begin_layout Plain Layout
975 line_parts = string.split(line, ':')
978 \begin_layout Plain Layout
980 num_str = line_parts[1]
983 \begin_layout Plain Layout
985 num_len = len(num_str)
988 \begin_layout Plain Layout
993 \begin_layout Plain Layout
995 while i < num_len and (num_str[i] in string.digits): i = i + 1
998 \begin_layout Plain Layout
1003 \begin_layout Plain Layout
1005 write_error(":" + line_parts[2], "noweb", int(num_str))
1008 \begin_layout Plain Layout
1013 \begin_layout Plain Layout
1023 \begin_layout Standard
1024 Some noweb messages are simply about undefined chunks.
1025 These can be seen by looking for matching double-angle-brackets.
1028 \begin_layout Standard
1032 \begin_layout Plain Layout
1034 <<Look for anything with double angle brackets>>=
1037 \begin_layout Plain Layout
1041 \begin_layout Plain Layout
1046 \begin_layout Plain Layout
1048 left = string.find(line, "<<")
1051 \begin_layout Plain Layout
1053 if (left != -1) and ((left + 2) < len(line)) and
1058 \begin_layout Plain Layout
1060 (string.find(line[left+2:], ">>") != -1):
1063 \begin_layout Plain Layout
1065 write_error(line, "noweb");
1068 \begin_layout Plain Layout
1073 \begin_layout Plain Layout
1083 \begin_layout Standard
1084 Finally, here is an additional list of explicit strings to check for.
1087 \begin_layout Standard
1091 \begin_layout Plain Layout
1093 <<Last ditch effort scan for specific strings>>=
1096 \begin_layout Plain Layout
1101 \begin_layout Plain Layout
1103 msgs_to_try = ("couldn't open file",
1106 \begin_layout Plain Layout
1108 "couldn't open temporary file",
1111 \begin_layout Plain Layout
1113 "error writing temporary file",
1116 \begin_layout Plain Layout
1118 "ill-formed option",
1121 \begin_layout Plain Layout
1126 \begin_layout Plain Layout
1128 "Bad format sequence",
1131 \begin_layout Plain Layout
1133 "Can't open output file",
1136 \begin_layout Plain Layout
1138 "Can't open temporary file",
1141 \begin_layout Plain Layout
1143 "Capacity exceeded:",
1146 \begin_layout Plain Layout
1148 "Ignoring unknown option -",
1151 \begin_layout Plain Layout
1153 "This can't happen:",
1156 \begin_layout Plain Layout
1158 "non-numeric line number in")
1161 \begin_layout Plain Layout
1163 for msg in msgs_to_try:
1166 \begin_layout Plain Layout
1168 if string.find(line, msg) != -1:
1171 \begin_layout Plain Layout
1173 write_error(line, "noweb")
1176 \begin_layout Plain Layout
1181 \begin_layout Plain Layout
1186 \begin_layout Plain Layout
1196 \begin_layout Subsection
1200 \begin_layout Standard
1201 The gcc errors can be multi-line, with the following format:
1204 \begin_layout LyX-Code
1205 foo.c: In function `main':
1208 \begin_layout Standard
1212 \begin_layout Plain Layout
1214 foo.c:3: `bar' undeclared (first use in this function)
1217 \begin_layout Plain Layout
1219 foo.c:3: (Each undeclared identifier is reported only once
1222 \begin_layout Plain Layout
1226 \begin_layout Plain Layout
1228 foo.c:3: for each function it appears in.)
1231 \begin_layout Plain Layout
1233 foo.c:3: parse error before `x'
1241 \begin_layout Standard
1242 In order to parse this, the gcc error handler has to look ahead and return
1243 any and all lines that do not match the expected pattern.
1246 \begin_layout Standard
1250 \begin_layout Plain Layout
1252 <<Function Bodies>>=
1255 \begin_layout Plain Layout
1260 \begin_layout Plain Layout
1262 """See if line is a gcc error.
1263 Read ahead to handle all the lines.
1266 \begin_layout Plain Layout
1270 \begin_layout Plain Layout
1272 Returns 1 on success, 0 otherwise.
1273 Outputs on stdout."""
1276 \begin_layout Plain Layout
1281 \begin_layout Plain Layout
1283 <<Handle the gcc error message>>
1286 \begin_layout Plain Layout
1291 \begin_layout Plain Layout
1295 \begin_layout Plain Layout
1305 \begin_layout Standard
1306 The error message starts with a gcc header (as above) without an associated
1310 \begin_layout Standard
1314 \begin_layout Plain Layout
1316 <<Handle the gcc error message>>=
1319 \begin_layout Plain Layout
1321 first_space = string.find(line, ' ')
1324 \begin_layout Plain Layout
1326 if first_space > 1: # The smallest would be "X: "
1329 \begin_layout Plain Layout
1331 if line[first_space - 1] == ':':
1334 \begin_layout Plain Layout
1336 header_to_see = line[:first_space - 1]
1339 \begin_layout Plain Layout
1341 next_line = getline()
1344 \begin_layout Plain Layout
1346 if next_line and next_line[:first_space - 1] == header_to_see:
1349 \begin_layout Plain Layout
1351 num_end = first_space
1354 \begin_layout Plain Layout
1356 while next_line[num_end] in string.digits: num_end = num_end + 1
1359 \begin_layout Plain Layout
1361 if num_end > first_space: # good!
1364 \begin_layout Plain Layout
1366 <<Accumulate gcc error lines and print it>>
1369 \begin_layout Plain Layout
1371 else: # oops! Not a gcc error.
1374 \begin_layout Plain Layout
1379 \begin_layout Plain Layout
1384 \begin_layout Plain Layout
1386 pushline(next_line) # return this line to input stream
1389 \begin_layout Plain Layout
1399 \begin_layout Standard
1400 At the point in the code that we know that we are in the middle of an error
1401 message, we do the following:
1404 \begin_layout Standard
1408 \begin_layout Plain Layout
1410 <<Accumulate gcc error lines and print it>>=
1413 \begin_layout Plain Layout
1415 num_str = next_line[first_space:num_end]
1418 \begin_layout Plain Layout
1420 msgs = [line[first_space:]]
1423 \begin_layout Plain Layout
1425 msgs.append(next_line[num_end + 1:])
1428 \begin_layout Plain Layout
1430 header_to_see = next_line[:num_end]
1433 \begin_layout Plain Layout
1435 next_line = getline()
1438 \begin_layout Plain Layout
1440 while next_line and next_line[:num_end] == header_to_see:
1443 \begin_layout Plain Layout
1445 msgs.append(next_line[num_end + 1:])
1448 \begin_layout Plain Layout
1450 next_line = getline()
1453 \begin_layout Plain Layout
1455 if next_line: pushline(next_line)
1458 \begin_layout Plain Layout
1460 write_error(msgs, "gcc", int(num_str))
1463 \begin_layout Plain Layout
1468 \begin_layout Plain Layout
1478 \begin_layout Subsection
1482 \begin_layout Standard
1483 A xlc error message is easy to identify.
1484 Every error message starts with a quoted string with no spaces, a comma,
1486 \begin_inset Quotes eld
1490 \begin_inset Quotes erd
1493 , a space, and some variable text.
1494 The following routine tests if a given buffer line matches this criteria
1495 (this code would probably be simpler if I used the
1496 \begin_inset Quotes eld
1500 \begin_inset Quotes erd
1503 regexp module, but we don't really need the full regular expression engine
1508 \begin_layout Standard
1512 \begin_layout Plain Layout
1516 \begin_layout Plain Layout
1520 \begin_layout Plain Layout
1522 <<Function Bodies>>=
1525 \begin_layout Plain Layout
1530 \begin_layout Plain Layout
1532 """see if line is an xlc error.
1535 \begin_layout Plain Layout
1539 \begin_layout Plain Layout
1541 Returns 1 on success, 0 otherwise.
1542 Outputs on stdout."""
1545 \begin_layout Plain Layout
1550 \begin_layout Plain Layout
1552 if line[0] == '"': # This is the first character of all xlc errors
1555 \begin_layout Plain Layout
1557 next_quote = string.find(line, '"', 1)
1560 \begin_layout Plain Layout
1562 first_space = string.find(line, ' ')
1565 \begin_layout Plain Layout
1567 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1571 \begin_layout Plain Layout
1573 if line[first_space - 1:first_space + 6] == ", line ":
1576 \begin_layout Plain Layout
1578 num_start = num_end = first_space + 6
1581 \begin_layout Plain Layout
1583 while line[num_end] in string.digits: num_end = num_end + 1
1586 \begin_layout Plain Layout
1588 if num_end > num_start:
1591 \begin_layout Plain Layout
1593 write_error(line, "xlc", int(line[num_start : num_end]))
1596 \begin_layout Plain Layout
1601 \begin_layout Plain Layout
1606 \begin_layout Plain Layout
1611 \begin_layout Plain Layout
1621 \begin_layout Section
1625 \begin_layout Standard
1626 This project can be tangled from \SpecialChar LyX
1628 \begin_inset Quotes eld
1632 \begin_inset Quotes erd
1635 convertor to call a generic script that always extracts a chunk named
1640 Here is an example of such a generic script:
1643 \begin_layout LyX-Code
1647 \begin_layout LyX-Code
1648 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 NOWEB_OUTPUT_DIR=$2 sh
1651 \begin_layout Standard
1652 This section defines our build-script, which extracts the code.
1655 \begin_layout Standard
1659 \begin_layout Plain Layout
1664 \begin_layout Plain Layout
1669 \begin_layout Plain Layout
1671 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1674 \begin_layout Plain Layout
1676 if [ -z "$NOWEB_OUTPUT_DIR" ]; then NOWEB_OUTPUT_DIR=.; fi
1679 \begin_layout Plain Layout
1681 notangle -Rlisterrors ${NOWEB_SOURCE} > ${NOWEB_OUTPUT_DIR}/listerrors
1684 \begin_layout Plain Layout
1686 chmod +x ${NOWEB_OUTPUT_DIR}/listerrors
1689 \begin_layout Plain Layout
1699 \begin_layout Section
1703 \begin_layout Standard
1704 This section provides cross-references into the rest of the program.
1707 \begin_layout Subsection
1711 \begin_layout Standard
1715 \begin_layout Plain Layout
1727 \begin_layout Subsection
1731 \begin_layout Standard
1735 \begin_layout Plain Layout