1 #LyX 2.1 created this file. For more info see http://www.lyx.org/
10 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
11 \@savsf=1% This is to get around a hyperref+noweb interaction problem
15 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
17 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
18 \let\nwdocspar=\smallbreak
20 \use_default_options false
24 \maintain_unincluded_children false
26 \language_package default
31 \font_typewriter courier
33 \font_default_family default
34 \use_non_tex_fonts false
40 \default_output_format pdf
42 \bibtex_command default
43 \index_command default
44 \paperfontsize default
49 \use_package amsmath 0
50 \use_package amssymb 0
53 \use_package mathdots 1
54 \use_package mathtools 0
56 \use_package stackrel 0
57 \use_package stmaryrd 0
58 \use_package undertilde 0
60 \cite_engine_type default
64 \paperorientation portrait
74 \paragraph_separation indent
75 \paragraph_indentation default
76 \quotes_language english
79 \paperpagestyle default
80 \tracking_changes false
91 \begin_inset Newline newline
100 \begin_inset Newline newline
104 \begin_inset Flex URL
107 \begin_layout Plain Layout
109 mailto:kayvan@sylvan.com
121 \begin_layout Abstract
122 The listerrors program used to be compiled as a C program and installed
127 along with LyX in order to perform some simple re-formatting of noweb and
129 This document describes and implements the Python version of the same program.
132 \begin_layout Standard
133 \begin_inset CommandInset toc
134 LatexCommand tableofcontents
141 \begin_layout Section
145 \begin_layout Standard
146 The motivation for this program was LyX bug 190
150 \begin_layout Plain Layout
152 \begin_inset Flex URL
155 \begin_layout Plain Layout
157 http://www.lyx.org/trac/ticket/190
168 \begin_inset Quotes eld
172 \begin_inset Quotes erd
178 \begin_layout Standard
180 \begin_inset Quotes eld
184 \begin_inset Quotes erd
187 ? Usually, LyX has great support for parsing of error messages.
188 For each error in the log file, LyX pops up an error box at that location
190 The error scanning routines expect these errors to be in a certain format
191 (similar to LaTeX errors).
192 When dealing with Literate Programs, you have
193 \begin_inset Quotes eld
200 \begin_layout Plain Layout
202 \begin_inset Flex URL
205 \begin_layout Plain Layout
207 http://www.eecs.harvard.edu/~nr/noweb
212 for more information about noweb.
218 \begin_inset Quotes erd
221 as well as gcc error messages (and potentially others).
222 The listerrors program attempts to standardize these error messages to
223 a format that LyX can parse and react to.
226 \begin_layout Standard
227 In a nutshell, the problems with the old implementation of listerrors that
228 bug 190 refers to were::
231 \begin_layout Enumerate
232 It was a C program and it was installed in the user path in the same directory
234 Having such a generically named binary in, for example,
238 , was potentially confusing.
241 \begin_layout Enumerate
242 It required that noweb be installed on the compiling machine (the source
243 was extracted by noweb from
245 SRCDIR/examples/Literate.lyx
247 , compiled and installed by
248 \begin_inset Quotes eld
252 \begin_inset Quotes erd
258 \begin_layout Standard
259 The new version deals with these problems in the following fashion:
262 \begin_layout Enumerate
263 Both the example file (this document) and the program are to be added to
264 the LyX CVS repository.
267 \begin_layout Enumerate
268 The program itself will be installed in
272 , along with other LyX-specific helper scripts.
275 \begin_layout Standard
276 In the design and implementation of this new
277 \begin_inset Quotes eld
281 \begin_inset Quotes erd
288 \begin_layout Plain Layout
289 See the Python home page (
290 \begin_inset Flex URL
293 \begin_layout Plain Layout
295 http://www.python.org
300 ) for more information.
305 language was chosen since it is fully multi-platform and provides a very
306 uniform and easy to read syntax.
307 This re-write also simplifies the code for
308 \begin_inset Quotes eld
312 \begin_inset Quotes erd
316 Python is installed by default on all modern Linux systems and is freely
317 available for all other platforms.
320 \begin_layout Standard
324 \begin_layout Plain Layout
329 \begin_layout Plain Layout
331 #!/usr/bin/python -tt
334 \begin_layout Plain Layout
336 """reformat noweb and compiler errors for LyX.
339 \begin_layout Plain Layout
343 \begin_layout Plain Layout
345 Expects to read from stdin and output to stdout.
348 \begin_layout Plain Layout
353 \begin_layout Plain Layout
357 \begin_layout Plain Layout
359 __author__ = "Kayvan A.
360 Sylvan <kayvan@sylvan.com>"
363 \begin_layout Plain Layout
365 __date__ = "$Date: 2005/07/18 09:42:26 $"
368 \begin_layout Plain Layout
370 __version__ = "$Revision: 1.5 $"
373 \begin_layout Plain Layout
375 __credits__ = """Edmar Wienskoski Jr.
376 <edmar-w-jr@technologist.com>
379 \begin_layout Plain Layout
381 original Literate support for LyX.
384 \begin_layout Plain Layout
386 Bernard Michael Hurley <berhardh@westherts.ac.uk>
389 \begin_layout Plain Layout
391 modifications to original listerrors."""
394 \begin_layout Plain Layout
396 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
399 \begin_layout Plain Layout
403 \begin_layout Plain Layout
408 \begin_layout Plain Layout
412 \begin_layout Plain Layout
417 \begin_layout Plain Layout
421 \begin_layout Plain Layout
423 if __name__ == "__main__":
426 \begin_layout Plain Layout
431 \begin_layout Plain Layout
441 \begin_layout Section
442 LaTeX style error message
445 \begin_layout Standard
446 The following function mimics the TeX error message format.
449 \begin_layout Standard
453 \begin_layout Plain Layout
458 \begin_layout Plain Layout
460 def write_error(msg, tool = "noweb", line_number = 1):
463 \begin_layout Plain Layout
465 """Write out the given message in TeX error style.
468 \begin_layout Plain Layout
472 \begin_layout Plain Layout
476 \begin_layout Plain Layout
478 called like: write_error(msg, tool, line_number)."""
481 \begin_layout Plain Layout
483 print "! Build Error: ==> %s ==>
488 \begin_layout Plain Layout
499 \begin_layout Plain Layout
501 if type(msg) == type("str"): # simple string
504 \begin_layout Plain Layout
509 \begin_layout Plain Layout
511 else: # some kind of list (sequence or tuple)
514 \begin_layout Plain Layout
519 \begin_layout Plain Layout
524 \begin_layout Plain Layout
529 \begin_layout Plain Layout
533 \begin_layout Plain Layout
543 \begin_layout Section
547 \begin_layout Standard
548 The only complication in our filtering code is that some parsers might need
549 to push back lines that are read in to be read again later.
550 We solve this problem by implementing a
551 \begin_inset Quotes eld
555 \begin_inset Quotes erd
559 \begin_inset Quotes eld
563 \begin_inset Quotes erd
569 \begin_layout Standard
573 \begin_layout Plain Layout
578 \begin_layout Plain Layout
580 __lines = [] # lines pushed back
583 \begin_layout Plain Layout
587 \begin_layout Plain Layout
589 def getline(file = sys.stdin):
592 \begin_layout Plain Layout
594 """read a line from internal stack or from file.
597 \begin_layout Plain Layout
601 \begin_layout Plain Layout
603 optional file argument defaults to sys.stdin."""
606 \begin_layout Plain Layout
611 \begin_layout Plain Layout
616 \begin_layout Plain Layout
621 \begin_layout Plain Layout
626 \begin_layout Plain Layout
631 \begin_layout Plain Layout
633 line = file.readline()
636 \begin_layout Plain Layout
641 \begin_layout Plain Layout
645 \begin_layout Plain Layout
655 \begin_layout Standard
656 And now for the corresponding pushline function:
659 \begin_layout Standard
663 \begin_layout Plain Layout
668 \begin_layout Plain Layout
673 \begin_layout Plain Layout
675 "push a line onto the pushback stack."
678 \begin_layout Plain Layout
682 \begin_layout Plain Layout
687 \begin_layout Plain Layout
692 \begin_layout Plain Layout
697 \begin_layout Plain Layout
701 \begin_layout Plain Layout
711 \begin_layout Standard
712 The main() entry point function is extremely simple.
713 Note that this version of
714 \begin_inset Quotes eld
718 \begin_inset Quotes erd
721 takes no options and simply filters, attempting simply to match against
722 the known error message patterns.
723 The listerrors C program handled a single-character command-line argument
724 that the current code no longer needs.
728 \begin_layout Standard
732 \begin_layout Plain Layout
737 \begin_layout Plain Layout
742 \begin_layout Plain Layout
744 """Entry point for listerrors.
748 \begin_layout Plain Layout
752 \begin_layout Plain Layout
754 Reads stdin and writes to stdout.
758 \begin_layout Plain Layout
762 \begin_layout Plain Layout
767 \begin_layout Plain Layout
772 \begin_layout Plain Layout
777 \begin_layout Plain Layout
779 <<Check line against patterns and take action>>
782 \begin_layout Plain Layout
792 \begin_layout Standard
793 For each line read in, we need to find out if it matches any of our tools
794 (noweb, gcc, etc.) and act accordingly.
797 \begin_layout Standard
801 \begin_layout Plain Layout
803 <<Check line against patterns and take action>>=
806 \begin_layout Plain Layout
810 \begin_layout Plain Layout
812 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
815 \begin_layout Plain Layout
817 for predicate in try_patterns_dispatch:
820 \begin_layout Plain Layout
822 if predicate(line): break
825 \begin_layout Plain Layout
835 \begin_layout Section
836 Different Error Formats
839 \begin_layout Standard
840 The following sections handle the various error message formats that we
841 recognize in this program.
845 \begin_layout Subsection
849 \begin_layout Standard
850 Noweb errors are output on a single line, so examining just the current
854 \begin_layout Standard
858 \begin_layout Plain Layout
863 \begin_layout Plain Layout
868 \begin_layout Plain Layout
870 """see if line is a noweb error.
873 \begin_layout Plain Layout
877 \begin_layout Plain Layout
879 Returns 1 on success, 0 otherwise.
880 Outputs on stdout."""
883 \begin_layout Plain Layout
888 \begin_layout Plain Layout
890 <<Look for the unescaped angle-brackets in documentation>>
893 \begin_layout Plain Layout
895 <<Look for anything with double angle brackets>>
898 \begin_layout Plain Layout
900 <<Last ditch effort scan for specific strings>>
903 \begin_layout Plain Layout
908 \begin_layout Plain Layout
912 \begin_layout Plain Layout
922 \begin_layout Standard
923 First, we look for the
924 \begin_inset Quotes eld
927 unescaped < < in documentation chunk
928 \begin_inset Quotes erd
932 This is the only message with an associated line number from noweb.
935 \begin_layout Standard
939 \begin_layout Plain Layout
943 \begin_layout Plain Layout
945 <<Look for the unescaped angle-brackets in documentation>>=
948 \begin_layout Plain Layout
950 if string.find(line, ": unescaped << in documentation chunk") != -1:
953 \begin_layout Plain Layout
955 line_parts = string.split(line, ':')
958 \begin_layout Plain Layout
960 num_str = line_parts[1]
963 \begin_layout Plain Layout
965 num_len = len(num_str)
968 \begin_layout Plain Layout
973 \begin_layout Plain Layout
975 while i < num_len and (num_str[i] in string.digits): i = i + 1
978 \begin_layout Plain Layout
983 \begin_layout Plain Layout
985 write_error(":" + line_parts[2], "noweb", int(num_str))
988 \begin_layout Plain Layout
993 \begin_layout Plain Layout
1003 \begin_layout Standard
1004 Some noweb messages are simply about undefined chunks.
1005 These can be seen by looking for matching double-angle-brackets.
1008 \begin_layout Standard
1012 \begin_layout Plain Layout
1014 <<Look for anything with double angle brackets>>=
1017 \begin_layout Plain Layout
1021 \begin_layout Plain Layout
1026 \begin_layout Plain Layout
1028 left = string.find(line, "<<")
1031 \begin_layout Plain Layout
1033 if (left != -1) and ((left + 2) < len(line)) and
1038 \begin_layout Plain Layout
1040 (string.find(line[left+2:], ">>") != -1):
1043 \begin_layout Plain Layout
1045 write_error(line, "noweb");
1048 \begin_layout Plain Layout
1053 \begin_layout Plain Layout
1063 \begin_layout Standard
1064 Finally, here is an additional list of explicit strings to check for.
1067 \begin_layout Standard
1071 \begin_layout Plain Layout
1073 <<Last ditch effort scan for specific strings>>=
1076 \begin_layout Plain Layout
1081 \begin_layout Plain Layout
1083 msgs_to_try = ("couldn't open file",
1086 \begin_layout Plain Layout
1088 "couldn't open temporary file",
1091 \begin_layout Plain Layout
1093 "error writing temporary file",
1096 \begin_layout Plain Layout
1098 "ill-formed option",
1101 \begin_layout Plain Layout
1106 \begin_layout Plain Layout
1108 "Bad format sequence",
1111 \begin_layout Plain Layout
1113 "Can't open output file",
1116 \begin_layout Plain Layout
1118 "Can't open temporary file",
1121 \begin_layout Plain Layout
1123 "Capacity exceeded:",
1126 \begin_layout Plain Layout
1128 "Ignoring unknown option -",
1131 \begin_layout Plain Layout
1133 "This can't happen:",
1136 \begin_layout Plain Layout
1138 "non-numeric line number in")
1141 \begin_layout Plain Layout
1143 for msg in msgs_to_try:
1146 \begin_layout Plain Layout
1148 if string.find(line, msg) != -1:
1151 \begin_layout Plain Layout
1153 write_error(line, "noweb")
1156 \begin_layout Plain Layout
1161 \begin_layout Plain Layout
1166 \begin_layout Plain Layout
1176 \begin_layout Subsection
1180 \begin_layout Standard
1181 The gcc errors can be multi-line, with the following format:
1184 \begin_layout LyX-Code
1185 foo.c: In function `main':
1188 \begin_layout Standard
1192 \begin_layout Plain Layout
1194 foo.c:3: `bar' undeclared (first use in this function)
1197 \begin_layout Plain Layout
1199 foo.c:3: (Each undeclared identifier is reported only once
1202 \begin_layout Plain Layout
1206 \begin_layout Plain Layout
1208 foo.c:3: for each function it appears in.)
1211 \begin_layout Plain Layout
1213 foo.c:3: parse error before `x'
1221 \begin_layout Standard
1222 In order to parse this, the gcc error handler has to look ahead and return
1223 any and all lines that do not match the expected pattern.
1226 \begin_layout Standard
1230 \begin_layout Plain Layout
1232 <<Function Bodies>>=
1235 \begin_layout Plain Layout
1240 \begin_layout Plain Layout
1242 """See if line is a gcc error.
1243 Read ahead to handle all the lines.
1246 \begin_layout Plain Layout
1250 \begin_layout Plain Layout
1252 Returns 1 on success, 0 otherwise.
1253 Outputs on stdout."""
1256 \begin_layout Plain Layout
1261 \begin_layout Plain Layout
1263 <<Handle the gcc error message>>
1266 \begin_layout Plain Layout
1271 \begin_layout Plain Layout
1275 \begin_layout Plain Layout
1285 \begin_layout Standard
1286 The error message starts with a gcc header (as above) without an associated
1290 \begin_layout Standard
1294 \begin_layout Plain Layout
1296 <<Handle the gcc error message>>=
1299 \begin_layout Plain Layout
1301 first_space = string.find(line, ' ')
1304 \begin_layout Plain Layout
1306 if first_space > 1: # The smallest would be "X: "
1309 \begin_layout Plain Layout
1311 if line[first_space - 1] == ':':
1314 \begin_layout Plain Layout
1316 header_to_see = line[:first_space - 1]
1319 \begin_layout Plain Layout
1321 next_line = getline()
1324 \begin_layout Plain Layout
1326 if next_line and next_line[:first_space - 1] == header_to_see:
1329 \begin_layout Plain Layout
1331 num_end = first_space
1334 \begin_layout Plain Layout
1336 while next_line[num_end] in string.digits: num_end = num_end + 1
1339 \begin_layout Plain Layout
1341 if num_end > first_space: # good!
1344 \begin_layout Plain Layout
1346 <<Accumulate gcc error lines and print it>>
1349 \begin_layout Plain Layout
1351 else: # oops! Not a gcc error.
1354 \begin_layout Plain Layout
1359 \begin_layout Plain Layout
1364 \begin_layout Plain Layout
1366 pushline(next_line) # return this line to input stream
1369 \begin_layout Plain Layout
1379 \begin_layout Standard
1380 At the point in the code that we know that we are in the middle of an error
1381 message, we do the following:
1384 \begin_layout Standard
1388 \begin_layout Plain Layout
1390 <<Accumulate gcc error lines and print it>>=
1393 \begin_layout Plain Layout
1395 num_str = next_line[first_space:num_end]
1398 \begin_layout Plain Layout
1400 msgs = [line[first_space:]]
1403 \begin_layout Plain Layout
1405 msgs.append(next_line[num_end + 1:])
1408 \begin_layout Plain Layout
1410 header_to_see = next_line[:num_end]
1413 \begin_layout Plain Layout
1415 next_line = getline()
1418 \begin_layout Plain Layout
1420 while next_line and next_line[:num_end] == header_to_see:
1423 \begin_layout Plain Layout
1425 msgs.append(next_line[num_end + 1:])
1428 \begin_layout Plain Layout
1430 next_line = getline()
1433 \begin_layout Plain Layout
1435 if next_line: pushline(next_line)
1438 \begin_layout Plain Layout
1440 write_error(msgs, "gcc", int(num_str))
1443 \begin_layout Plain Layout
1448 \begin_layout Plain Layout
1458 \begin_layout Subsection
1462 \begin_layout Standard
1463 A xlc error message is easy to identify.
1464 Every error message starts with a quoted string with no spaces, a comma,
1466 \begin_inset Quotes eld
1470 \begin_inset Quotes erd
1473 , a space, and some variable text.
1474 The following routine tests if a given buffer line matches this criteria
1475 (this code would probably be simpler if I used the
1476 \begin_inset Quotes eld
1480 \begin_inset Quotes erd
1483 regexp module, but we don't really need the full regular expression engine
1488 \begin_layout Standard
1492 \begin_layout Plain Layout
1496 \begin_layout Plain Layout
1500 \begin_layout Plain Layout
1502 <<Function Bodies>>=
1505 \begin_layout Plain Layout
1510 \begin_layout Plain Layout
1512 """see if line is an xlc error.
1515 \begin_layout Plain Layout
1519 \begin_layout Plain Layout
1521 Returns 1 on success, 0 otherwise.
1522 Outputs on stdout."""
1525 \begin_layout Plain Layout
1530 \begin_layout Plain Layout
1532 if line[0] == '"': # This is the first character of all xlc errors
1535 \begin_layout Plain Layout
1537 next_quote = string.find(line, '"', 1)
1540 \begin_layout Plain Layout
1542 first_space = string.find(line, ' ')
1545 \begin_layout Plain Layout
1547 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1551 \begin_layout Plain Layout
1553 if line[first_space - 1:first_space + 6] == ", line ":
1556 \begin_layout Plain Layout
1558 num_start = num_end = first_space + 6
1561 \begin_layout Plain Layout
1563 while line[num_end] in string.digits: num_end = num_end + 1
1566 \begin_layout Plain Layout
1568 if num_end > num_start:
1571 \begin_layout Plain Layout
1573 write_error(line, "xlc", int(line[num_start : num_end]))
1576 \begin_layout Plain Layout
1581 \begin_layout Plain Layout
1586 \begin_layout Plain Layout
1591 \begin_layout Plain Layout
1601 \begin_layout Section
1605 \begin_layout Standard
1606 This project can be tangled from LyX if you set your
1607 \begin_inset Quotes eld
1611 \begin_inset Quotes erd
1614 convertor to call a generic script that always extracts a chunk named
1619 Here is an example of such a generic script:
1622 \begin_layout LyX-Code
1626 \begin_layout LyX-Code
1627 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 NOWEB_OUTPUT_DIR=$2 sh
1630 \begin_layout Standard
1631 This section defines our build-script, which extracts the code.
1634 \begin_layout Standard
1638 \begin_layout Plain Layout
1643 \begin_layout Plain Layout
1648 \begin_layout Plain Layout
1650 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1653 \begin_layout Plain Layout
1655 if [ -z "$NOWEB_OUTPUT_DIR" ]; then NOWEB_OUTPUT_DIR=.; fi
1658 \begin_layout Plain Layout
1660 notangle -Rlisterrors ${NOWEB_SOURCE} > ${NOWEB_OUTPUT_DIR}/listerrors
1663 \begin_layout Plain Layout
1665 chmod +x ${NOWEB_OUTPUT_DIR}/listerrors
1668 \begin_layout Plain Layout
1678 \begin_layout Section
1682 \begin_layout Standard
1683 This section provides cross-references into the rest of the program.
1686 \begin_layout Subsection
1690 \begin_layout Standard
1694 \begin_layout Plain Layout
1706 \begin_layout Subsection
1710 \begin_layout Standard
1714 \begin_layout Plain Layout