1 #LyX 2.2 created this file. For more info see http://www.lyx.org/
5 \save_transient_properties true
6 \origin /systemlyxdir/examples/
12 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
13 \@savsf=1% This is to get around a hyperref+noweb interaction problem
17 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
19 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
20 \let\nwdocspar=\smallbreak
22 \use_default_options false
26 \maintain_unincluded_children false
28 \language_package default
31 \font_roman "times" "default"
32 \font_sans "helvet" "default"
33 \font_typewriter "courier" "default"
34 \font_math "auto" "auto"
35 \font_default_family default
36 \use_non_tex_fonts false
39 \font_sf_scale 100 100
40 \font_tt_scale 100 100
42 \default_output_format pdf
44 \bibtex_command default
45 \index_command default
46 \paperfontsize default
51 \use_package amsmath 0
52 \use_package amssymb 0
55 \use_package mathdots 1
56 \use_package mathtools 0
58 \use_package stackrel 0
59 \use_package stmaryrd 0
60 \use_package undertilde 0
62 \cite_engine_type default
66 \paperorientation portrait
76 \paragraph_separation indent
77 \paragraph_indentation default
78 \quotes_language english
81 \paperpagestyle default
82 \tracking_changes false
94 \begin_inset Newline newline
103 \begin_inset Newline newline
107 \begin_inset Flex URL
110 \begin_layout Plain Layout
112 mailto:kayvan@sylvan.com
124 \begin_layout Abstract
125 The listerrors program used to be compiled as a C program and installed
130 along with \SpecialChar LyX
131 in order to perform some simple re-formatting of noweb and
133 This document describes and implements the Python version of the same program.
136 \begin_layout Standard
137 \begin_inset CommandInset toc
138 LatexCommand tableofcontents
145 \begin_layout Section
149 \begin_layout Standard
150 The motivation for this program was \SpecialChar LyX
155 \begin_layout Plain Layout
157 \begin_inset Flex URL
160 \begin_layout Plain Layout
162 http://www.lyx.org/trac/ticket/190
173 \begin_inset Quotes eld
177 \begin_inset Quotes erd
183 \begin_layout Standard
185 \begin_inset Quotes eld
189 \begin_inset Quotes erd
192 ? Usually, \SpecialChar LyX
193 has great support for parsing of error messages.
194 For each error in the log file, \SpecialChar LyX
195 pops up an error box at that location
196 in the \SpecialChar LyX
198 The error scanning routines expect these errors to be in a certain format
199 (similar to \SpecialChar LaTeX
201 When dealing with Literate Programs, you have
202 \begin_inset Quotes eld
209 \begin_layout Plain Layout
211 \begin_inset Flex URL
214 \begin_layout Plain Layout
216 http://www.eecs.harvard.edu/~nr/noweb
221 for more information about noweb.
227 \begin_inset Quotes erd
230 as well as gcc error messages (and potentially others).
231 The listerrors program attempts to standardize these error messages to
232 a format that \SpecialChar LyX
233 can parse and react to.
236 \begin_layout Standard
237 In a nutshell, the problems with the old implementation of listerrors that
238 bug 190 refers to were::
241 \begin_layout Enumerate
242 It was a C program and it was installed in the user path in the same directory
245 Having such a generically named binary in, for example,
249 , was potentially confusing.
252 \begin_layout Enumerate
253 It required that noweb be installed on the compiling machine (the source
254 was extracted by noweb from
256 SRCDIR/examples/Literate.lyx
258 , compiled and installed by
259 \begin_inset Quotes eld
263 \begin_inset Quotes erd
269 \begin_layout Standard
270 The new version deals with these problems in the following fashion:
273 \begin_layout Enumerate
274 Both the example file (this document) and the program are to be added to
279 \begin_layout Enumerate
280 The program itself will be installed in
284 , along with other \SpecialChar LyX
285 -specific helper scripts.
288 \begin_layout Standard
289 In the design and implementation of this new
290 \begin_inset Quotes eld
294 \begin_inset Quotes erd
301 \begin_layout Plain Layout
302 See the Python home page (
303 \begin_inset Flex URL
306 \begin_layout Plain Layout
308 http://www.python.org
313 ) for more information.
318 language was chosen since it is fully multi-platform and provides a very
319 uniform and easy to read syntax.
320 This re-write also simplifies the code for
321 \begin_inset Quotes eld
325 \begin_inset Quotes erd
329 Python is installed by default on all modern Linux systems and is freely
330 available for all other platforms.
333 \begin_layout Standard
337 \begin_layout Plain Layout
342 \begin_layout Plain Layout
344 #!/usr/bin/python -tt
347 \begin_layout Plain Layout
349 """reformat noweb and compiler errors for LyX.
352 \begin_layout Plain Layout
356 \begin_layout Plain Layout
358 Expects to read from stdin and output to stdout.
361 \begin_layout Plain Layout
366 \begin_layout Plain Layout
370 \begin_layout Plain Layout
372 __author__ = "Kayvan A.
373 Sylvan <kayvan@sylvan.com>"
376 \begin_layout Plain Layout
378 __date__ = "$Date: 2005/07/18 09:42:26 $"
381 \begin_layout Plain Layout
383 __version__ = "$Revision: 1.5 $"
386 \begin_layout Plain Layout
388 __credits__ = """Edmar Wienskoski Jr.
389 <edmar-w-jr@technologist.com>
392 \begin_layout Plain Layout
394 original Literate support for LyX.
397 \begin_layout Plain Layout
399 Bernard Michael Hurley <berhardh@westherts.ac.uk>
402 \begin_layout Plain Layout
404 modifications to original listerrors."""
407 \begin_layout Plain Layout
409 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
412 \begin_layout Plain Layout
416 \begin_layout Plain Layout
421 \begin_layout Plain Layout
425 \begin_layout Plain Layout
430 \begin_layout Plain Layout
434 \begin_layout Plain Layout
436 if __name__ == "__main__":
439 \begin_layout Plain Layout
444 \begin_layout Plain Layout
454 \begin_layout Section
459 \begin_layout Standard
460 The following function mimics the \SpecialChar TeX
461 error message format.
464 \begin_layout Standard
468 \begin_layout Plain Layout
473 \begin_layout Plain Layout
475 def write_error(msg, tool = "noweb", line_number = 1):
478 \begin_layout Plain Layout
480 """Write out the given message in TeX error style.
483 \begin_layout Plain Layout
487 \begin_layout Plain Layout
491 \begin_layout Plain Layout
493 called like: write_error(msg, tool, line_number)."""
496 \begin_layout Plain Layout
498 print "! Build Error: ==> %s ==>
503 \begin_layout Plain Layout
514 \begin_layout Plain Layout
516 if type(msg) == type("str"): # simple string
519 \begin_layout Plain Layout
524 \begin_layout Plain Layout
526 else: # some kind of list (sequence or tuple)
529 \begin_layout Plain Layout
534 \begin_layout Plain Layout
539 \begin_layout Plain Layout
544 \begin_layout Plain Layout
548 \begin_layout Plain Layout
558 \begin_layout Section
562 \begin_layout Standard
563 The only complication in our filtering code is that some parsers might need
564 to push back lines that are read in to be read again later.
565 We solve this problem by implementing a
566 \begin_inset Quotes eld
570 \begin_inset Quotes erd
574 \begin_inset Quotes eld
578 \begin_inset Quotes erd
584 \begin_layout Standard
588 \begin_layout Plain Layout
593 \begin_layout Plain Layout
595 __lines = [] # lines pushed back
598 \begin_layout Plain Layout
602 \begin_layout Plain Layout
604 def getline(file = sys.stdin):
607 \begin_layout Plain Layout
609 """read a line from internal stack or from file.
612 \begin_layout Plain Layout
616 \begin_layout Plain Layout
618 optional file argument defaults to sys.stdin."""
621 \begin_layout Plain Layout
626 \begin_layout Plain Layout
631 \begin_layout Plain Layout
636 \begin_layout Plain Layout
641 \begin_layout Plain Layout
646 \begin_layout Plain Layout
648 line = file.readline()
651 \begin_layout Plain Layout
656 \begin_layout Plain Layout
660 \begin_layout Plain Layout
670 \begin_layout Standard
671 And now for the corresponding pushline function:
674 \begin_layout Standard
678 \begin_layout Plain Layout
683 \begin_layout Plain Layout
688 \begin_layout Plain Layout
690 "push a line onto the pushback stack."
693 \begin_layout Plain Layout
697 \begin_layout Plain Layout
702 \begin_layout Plain Layout
707 \begin_layout Plain Layout
712 \begin_layout Plain Layout
716 \begin_layout Plain Layout
726 \begin_layout Standard
727 The main() entry point function is extremely simple.
728 Note that this version of
729 \begin_inset Quotes eld
733 \begin_inset Quotes erd
736 takes no options and simply filters, attempting simply to match against
737 the known error message patterns.
738 The listerrors C program handled a single-character command-line argument
739 that the current code no longer needs.
743 \begin_layout Standard
747 \begin_layout Plain Layout
752 \begin_layout Plain Layout
757 \begin_layout Plain Layout
759 """Entry point for listerrors.
763 \begin_layout Plain Layout
767 \begin_layout Plain Layout
769 Reads stdin and writes to stdout.
773 \begin_layout Plain Layout
777 \begin_layout Plain Layout
782 \begin_layout Plain Layout
787 \begin_layout Plain Layout
792 \begin_layout Plain Layout
794 <<Check line against patterns and take action>>
797 \begin_layout Plain Layout
807 \begin_layout Standard
808 For each line read in, we need to find out if it matches any of our tools
809 (noweb, gcc, etc.) and act accordingly.
812 \begin_layout Standard
816 \begin_layout Plain Layout
818 <<Check line against patterns and take action>>=
821 \begin_layout Plain Layout
825 \begin_layout Plain Layout
827 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
830 \begin_layout Plain Layout
832 for predicate in try_patterns_dispatch:
835 \begin_layout Plain Layout
837 if predicate(line): break
840 \begin_layout Plain Layout
850 \begin_layout Section
851 Different Error Formats
854 \begin_layout Standard
855 The following sections handle the various error message formats that we
856 recognize in this program.
860 \begin_layout Subsection
864 \begin_layout Standard
865 Noweb errors are output on a single line, so examining just the current
869 \begin_layout Standard
873 \begin_layout Plain Layout
878 \begin_layout Plain Layout
883 \begin_layout Plain Layout
885 """see if line is a noweb error.
888 \begin_layout Plain Layout
892 \begin_layout Plain Layout
894 Returns 1 on success, 0 otherwise.
895 Outputs on stdout."""
898 \begin_layout Plain Layout
903 \begin_layout Plain Layout
905 <<Look for the unescaped angle-brackets in documentation>>
908 \begin_layout Plain Layout
910 <<Look for anything with double angle brackets>>
913 \begin_layout Plain Layout
915 <<Last ditch effort scan for specific strings>>
918 \begin_layout Plain Layout
923 \begin_layout Plain Layout
927 \begin_layout Plain Layout
937 \begin_layout Standard
938 First, we look for the
939 \begin_inset Quotes eld
942 unescaped < < in documentation chunk
943 \begin_inset Quotes erd
947 This is the only message with an associated line number from noweb.
950 \begin_layout Standard
954 \begin_layout Plain Layout
958 \begin_layout Plain Layout
960 <<Look for the unescaped angle-brackets in documentation>>=
963 \begin_layout Plain Layout
965 if string.find(line, ": unescaped << in documentation chunk") != -1:
968 \begin_layout Plain Layout
970 line_parts = string.split(line, ':')
973 \begin_layout Plain Layout
975 num_str = line_parts[1]
978 \begin_layout Plain Layout
980 num_len = len(num_str)
983 \begin_layout Plain Layout
988 \begin_layout Plain Layout
990 while i < num_len and (num_str[i] in string.digits): i = i + 1
993 \begin_layout Plain Layout
998 \begin_layout Plain Layout
1000 write_error(":" + line_parts[2], "noweb", int(num_str))
1003 \begin_layout Plain Layout
1008 \begin_layout Plain Layout
1018 \begin_layout Standard
1019 Some noweb messages are simply about undefined chunks.
1020 These can be seen by looking for matching double-angle-brackets.
1023 \begin_layout Standard
1027 \begin_layout Plain Layout
1029 <<Look for anything with double angle brackets>>=
1032 \begin_layout Plain Layout
1036 \begin_layout Plain Layout
1041 \begin_layout Plain Layout
1043 left = string.find(line, "<<")
1046 \begin_layout Plain Layout
1048 if (left != -1) and ((left + 2) < len(line)) and
1053 \begin_layout Plain Layout
1055 (string.find(line[left+2:], ">>") != -1):
1058 \begin_layout Plain Layout
1060 write_error(line, "noweb");
1063 \begin_layout Plain Layout
1068 \begin_layout Plain Layout
1078 \begin_layout Standard
1079 Finally, here is an additional list of explicit strings to check for.
1082 \begin_layout Standard
1086 \begin_layout Plain Layout
1088 <<Last ditch effort scan for specific strings>>=
1091 \begin_layout Plain Layout
1096 \begin_layout Plain Layout
1098 msgs_to_try = ("couldn't open file",
1101 \begin_layout Plain Layout
1103 "couldn't open temporary file",
1106 \begin_layout Plain Layout
1108 "error writing temporary file",
1111 \begin_layout Plain Layout
1113 "ill-formed option",
1116 \begin_layout Plain Layout
1121 \begin_layout Plain Layout
1123 "Bad format sequence",
1126 \begin_layout Plain Layout
1128 "Can't open output file",
1131 \begin_layout Plain Layout
1133 "Can't open temporary file",
1136 \begin_layout Plain Layout
1138 "Capacity exceeded:",
1141 \begin_layout Plain Layout
1143 "Ignoring unknown option -",
1146 \begin_layout Plain Layout
1148 "This can't happen:",
1151 \begin_layout Plain Layout
1153 "non-numeric line number in")
1156 \begin_layout Plain Layout
1158 for msg in msgs_to_try:
1161 \begin_layout Plain Layout
1163 if string.find(line, msg) != -1:
1166 \begin_layout Plain Layout
1168 write_error(line, "noweb")
1171 \begin_layout Plain Layout
1176 \begin_layout Plain Layout
1181 \begin_layout Plain Layout
1191 \begin_layout Subsection
1195 \begin_layout Standard
1196 The gcc errors can be multi-line, with the following format:
1199 \begin_layout LyX-Code
1200 foo.c: In function `main':
1203 \begin_layout Standard
1207 \begin_layout Plain Layout
1209 foo.c:3: `bar' undeclared (first use in this function)
1212 \begin_layout Plain Layout
1214 foo.c:3: (Each undeclared identifier is reported only once
1217 \begin_layout Plain Layout
1221 \begin_layout Plain Layout
1223 foo.c:3: for each function it appears in.)
1226 \begin_layout Plain Layout
1228 foo.c:3: parse error before `x'
1236 \begin_layout Standard
1237 In order to parse this, the gcc error handler has to look ahead and return
1238 any and all lines that do not match the expected pattern.
1241 \begin_layout Standard
1245 \begin_layout Plain Layout
1247 <<Function Bodies>>=
1250 \begin_layout Plain Layout
1255 \begin_layout Plain Layout
1257 """See if line is a gcc error.
1258 Read ahead to handle all the lines.
1261 \begin_layout Plain Layout
1265 \begin_layout Plain Layout
1267 Returns 1 on success, 0 otherwise.
1268 Outputs on stdout."""
1271 \begin_layout Plain Layout
1276 \begin_layout Plain Layout
1278 <<Handle the gcc error message>>
1281 \begin_layout Plain Layout
1286 \begin_layout Plain Layout
1290 \begin_layout Plain Layout
1300 \begin_layout Standard
1301 The error message starts with a gcc header (as above) without an associated
1305 \begin_layout Standard
1309 \begin_layout Plain Layout
1311 <<Handle the gcc error message>>=
1314 \begin_layout Plain Layout
1316 first_space = string.find(line, ' ')
1319 \begin_layout Plain Layout
1321 if first_space > 1: # The smallest would be "X: "
1324 \begin_layout Plain Layout
1326 if line[first_space - 1] == ':':
1329 \begin_layout Plain Layout
1331 header_to_see = line[:first_space - 1]
1334 \begin_layout Plain Layout
1336 next_line = getline()
1339 \begin_layout Plain Layout
1341 if next_line and next_line[:first_space - 1] == header_to_see:
1344 \begin_layout Plain Layout
1346 num_end = first_space
1349 \begin_layout Plain Layout
1351 while next_line[num_end] in string.digits: num_end = num_end + 1
1354 \begin_layout Plain Layout
1356 if num_end > first_space: # good!
1359 \begin_layout Plain Layout
1361 <<Accumulate gcc error lines and print it>>
1364 \begin_layout Plain Layout
1366 else: # oops! Not a gcc error.
1369 \begin_layout Plain Layout
1374 \begin_layout Plain Layout
1379 \begin_layout Plain Layout
1381 pushline(next_line) # return this line to input stream
1384 \begin_layout Plain Layout
1394 \begin_layout Standard
1395 At the point in the code that we know that we are in the middle of an error
1396 message, we do the following:
1399 \begin_layout Standard
1403 \begin_layout Plain Layout
1405 <<Accumulate gcc error lines and print it>>=
1408 \begin_layout Plain Layout
1410 num_str = next_line[first_space:num_end]
1413 \begin_layout Plain Layout
1415 msgs = [line[first_space:]]
1418 \begin_layout Plain Layout
1420 msgs.append(next_line[num_end + 1:])
1423 \begin_layout Plain Layout
1425 header_to_see = next_line[:num_end]
1428 \begin_layout Plain Layout
1430 next_line = getline()
1433 \begin_layout Plain Layout
1435 while next_line and next_line[:num_end] == header_to_see:
1438 \begin_layout Plain Layout
1440 msgs.append(next_line[num_end + 1:])
1443 \begin_layout Plain Layout
1445 next_line = getline()
1448 \begin_layout Plain Layout
1450 if next_line: pushline(next_line)
1453 \begin_layout Plain Layout
1455 write_error(msgs, "gcc", int(num_str))
1458 \begin_layout Plain Layout
1463 \begin_layout Plain Layout
1473 \begin_layout Subsection
1477 \begin_layout Standard
1478 A xlc error message is easy to identify.
1479 Every error message starts with a quoted string with no spaces, a comma,
1481 \begin_inset Quotes eld
1485 \begin_inset Quotes erd
1488 , a space, and some variable text.
1489 The following routine tests if a given buffer line matches this criteria
1490 (this code would probably be simpler if I used the
1491 \begin_inset Quotes eld
1495 \begin_inset Quotes erd
1498 regexp module, but we don't really need the full regular expression engine
1503 \begin_layout Standard
1507 \begin_layout Plain Layout
1511 \begin_layout Plain Layout
1515 \begin_layout Plain Layout
1517 <<Function Bodies>>=
1520 \begin_layout Plain Layout
1525 \begin_layout Plain Layout
1527 """see if line is an xlc error.
1530 \begin_layout Plain Layout
1534 \begin_layout Plain Layout
1536 Returns 1 on success, 0 otherwise.
1537 Outputs on stdout."""
1540 \begin_layout Plain Layout
1545 \begin_layout Plain Layout
1547 if line[0] == '"': # This is the first character of all xlc errors
1550 \begin_layout Plain Layout
1552 next_quote = string.find(line, '"', 1)
1555 \begin_layout Plain Layout
1557 first_space = string.find(line, ' ')
1560 \begin_layout Plain Layout
1562 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1566 \begin_layout Plain Layout
1568 if line[first_space - 1:first_space + 6] == ", line ":
1571 \begin_layout Plain Layout
1573 num_start = num_end = first_space + 6
1576 \begin_layout Plain Layout
1578 while line[num_end] in string.digits: num_end = num_end + 1
1581 \begin_layout Plain Layout
1583 if num_end > num_start:
1586 \begin_layout Plain Layout
1588 write_error(line, "xlc", int(line[num_start : num_end]))
1591 \begin_layout Plain Layout
1596 \begin_layout Plain Layout
1601 \begin_layout Plain Layout
1606 \begin_layout Plain Layout
1616 \begin_layout Section
1620 \begin_layout Standard
1621 This project can be tangled from \SpecialChar LyX
1623 \begin_inset Quotes eld
1627 \begin_inset Quotes erd
1630 convertor to call a generic script that always extracts a chunk named
1635 Here is an example of such a generic script:
1638 \begin_layout LyX-Code
1642 \begin_layout LyX-Code
1643 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 NOWEB_OUTPUT_DIR=$2 sh
1646 \begin_layout Standard
1647 This section defines our build-script, which extracts the code.
1650 \begin_layout Standard
1654 \begin_layout Plain Layout
1659 \begin_layout Plain Layout
1664 \begin_layout Plain Layout
1666 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1669 \begin_layout Plain Layout
1671 if [ -z "$NOWEB_OUTPUT_DIR" ]; then NOWEB_OUTPUT_DIR=.; fi
1674 \begin_layout Plain Layout
1676 notangle -Rlisterrors ${NOWEB_SOURCE} > ${NOWEB_OUTPUT_DIR}/listerrors
1679 \begin_layout Plain Layout
1681 chmod +x ${NOWEB_OUTPUT_DIR}/listerrors
1684 \begin_layout Plain Layout
1694 \begin_layout Section
1698 \begin_layout Standard
1699 This section provides cross-references into the rest of the program.
1702 \begin_layout Subsection
1706 \begin_layout Standard
1710 \begin_layout Plain Layout
1722 \begin_layout Subsection
1726 \begin_layout Standard
1730 \begin_layout Plain Layout