1 #LyX 2.1 created this file. For more info see http://www.lyx.org/
10 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
11 \@savsf=1% This is to get around a hyperref+noweb interaction problem
15 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
17 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
18 \let\nwdocspar=\smallbreak
20 \use_default_options false
24 \maintain_unincluded_children false
26 \language_package default
31 \font_typewriter courier
33 \font_default_family default
34 \use_non_tex_fonts false
40 \default_output_format pdf
42 \bibtex_command default
43 \index_command default
44 \paperfontsize default
49 \use_package amsmath 0
50 \use_package amssymb 0
52 \use_package mathdots 1
53 \use_package mathtools 0
55 \use_package undertilde 0
57 \cite_engine_type numerical
61 \paperorientation portrait
71 \paragraph_separation indent
72 \paragraph_indentation default
73 \quotes_language english
76 \paperpagestyle default
77 \tracking_changes false
88 \begin_inset Newline newline
97 \begin_inset Newline newline
101 \begin_inset Flex URL
104 \begin_layout Plain Layout
106 mailto:kayvan@sylvan.com
118 \begin_layout Abstract
119 The listerrors program used to be compiled as a C program and installed
124 along with LyX in order to perform some simple re-formatting of noweb and
126 This document describes and implements the Python version of the same program.
129 \begin_layout Standard
130 \begin_inset CommandInset toc
131 LatexCommand tableofcontents
138 \begin_layout Section
142 \begin_layout Standard
143 The motivation for this program was LyX bug 190
147 \begin_layout Plain Layout
149 \begin_inset Flex URL
152 \begin_layout Plain Layout
154 http://www.lyx.org/trac/ticket/190
165 \begin_inset Quotes eld
169 \begin_inset Quotes erd
175 \begin_layout Standard
177 \begin_inset Quotes eld
181 \begin_inset Quotes erd
184 ? Usually, LyX has great support for parsing of error messages.
185 For each error in the log file, LyX pops up an error box at that location
187 The error scanning routines expect these errors to be in a certain format
188 (similar to LaTeX errors).
189 When dealing with Literate Programs, you have
190 \begin_inset Quotes eld
197 \begin_layout Plain Layout
199 \begin_inset Flex URL
202 \begin_layout Plain Layout
204 http://www.eecs.harvard.edu/~nr/noweb
209 for more information about noweb.
215 \begin_inset Quotes erd
218 as well as gcc error messages (and potentially others).
219 The listerrors program attempts to standardize these error messages to
220 a format that LyX can parse and react to.
223 \begin_layout Standard
224 In a nutshell, the problems with the old implementation of listerrors that
225 bug 190 refers to were::
228 \begin_layout Enumerate
229 It was a C program and it was installed in the user path in the same directory
231 Having such a generically named binary in, for example,
235 , was potentially confusing.
238 \begin_layout Enumerate
239 It required that noweb be installed on the compiling machine (the source
240 was extracted by noweb from
242 SRCDIR/examples/Literate.lyx
244 , compiled and installed by
245 \begin_inset Quotes eld
249 \begin_inset Quotes erd
255 \begin_layout Standard
256 The new version deals with these problems in the following fashion:
259 \begin_layout Enumerate
260 Both the example file (this document) and the program are to be added to
261 the LyX CVS repository.
264 \begin_layout Enumerate
265 The program itself will be installed in
269 , along with other LyX-specific helper scripts.
272 \begin_layout Standard
273 In the design and implementation of this new
274 \begin_inset Quotes eld
278 \begin_inset Quotes erd
285 \begin_layout Plain Layout
286 See the Python home page (
287 \begin_inset Flex URL
290 \begin_layout Plain Layout
292 http://www.python.org
297 ) for more information.
302 language was chosen since it is fully multi-platform and provides a very
303 uniform and easy to read syntax.
304 This re-write also simplifies the code for
305 \begin_inset Quotes eld
309 \begin_inset Quotes erd
313 Python is installed by default on all modern Linux systems and is freely
314 available for all other platforms.
324 #!/usr/bin/python -tt
329 """reformat noweb and compiler errors for LyX.
338 Expects to read from stdin and output to stdout.
352 __author__ = "Kayvan A.
353 Sylvan <kayvan@sylvan.com>"
358 __date__ = "$Date: 2005/07/18 09:42:26 $"
363 __version__ = "$Revision: 1.5 $"
368 __credits__ = """Edmar Wienskoski Jr.
369 <edmar-w-jr@technologist.com>
374 original Literate support for LyX.
379 Bernard Michael Hurley <berhardh@westherts.ac.uk>
384 modifications to original listerrors."""
389 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
416 if __name__ == "__main__":
429 \begin_layout Section
430 LaTeX style error message
433 \begin_layout Standard
434 The following function mimics the TeX error message format.
444 def write_error(msg, tool = "noweb", line_number = 1):
449 """Write out the given message in TeX error style.
462 called like: write_error(msg, tool, line_number)."""
463 \begin_inset Newline newline
466 print "! Build Error: ==> %s ==>
469 \begin_inset Newline newline
479 \begin_inset Newline newline
482 if type(msg) == type("str"): # simple string
483 \begin_inset Newline newline
487 \begin_inset Newline newline
490 else: # some kind of list (sequence or tuple)
491 \begin_inset Newline newline
495 \begin_inset Newline newline
499 \begin_inset Newline newline
503 \begin_inset Newline newline
507 \begin_inset Newline newline
513 \begin_layout Section
517 \begin_layout Standard
518 The only complication in our filtering code is that some parsers might need
519 to push back lines that are read in to be read again later.
520 We solve this problem by implementing a
521 \begin_inset Quotes eld
525 \begin_inset Quotes erd
529 \begin_inset Quotes eld
533 \begin_inset Quotes erd
546 __lines = [] # lines pushed back
555 def getline(file = sys.stdin):
560 """read a line from internal stack or from file.
569 optional file argument defaults to sys.stdin."""
599 line = file.readline()
616 \begin_layout Standard
617 And now for the corresponding pushline function:
632 "push a line onto the pushback stack."
642 \begin_inset Newline newline
646 \begin_inset Newline newline
650 \begin_inset Newline newline
654 \begin_inset Newline newline
660 \begin_layout Standard
661 The main() entry point function is extremely simple.
662 Note that this version of
663 \begin_inset Quotes eld
667 \begin_inset Quotes erd
670 takes no options and simply filters, attempting simply to match against
671 the known error message patterns.
672 The listerrors C program handled a single-character command-line argument
673 that the current code no longer needs.
689 """Entry point for listerrors.
699 Reads stdin and writes to stdout.
724 <<Check line against patterns and take action>>
732 \begin_layout Standard
733 For each line read in, we need to find out if it matches any of our tools
734 (noweb, gcc, etc.) and act accordingly.
739 <<Check line against patterns and take action>>=
748 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
749 \begin_inset Newline newline
752 for predicate in try_patterns_dispatch:
753 \begin_inset Newline newline
756 if predicate(line): break
757 \begin_inset Newline newline
763 \begin_layout Section
764 Different Error Formats
767 \begin_layout Standard
768 The following sections handle the various error message formats that we
769 recognize in this program.
773 \begin_layout Subsection
777 \begin_layout Standard
778 Noweb errors are output on a single line, so examining just the current
794 """see if line is a noweb error.
803 Returns 1 on success, 0 otherwise.
804 Outputs on stdout."""
814 <<Look for the unescaped angle-brackets in documentation>>
819 <<Look for anything with double angle brackets>>
824 <<Last ditch effort scan for specific strings>>
841 \begin_layout Standard
842 First, we look for the
843 \begin_inset Quotes eld
846 unescaped < < in documentation chunk
847 \begin_inset Quotes erd
851 This is the only message with an associated line number from noweb.
860 <<Look for the unescaped angle-brackets in documentation>>=
865 if string.find(line, ": unescaped << in documentation chunk") != -1:
870 line_parts = string.split(line, ':')
875 num_str = line_parts[1]
880 num_len = len(num_str)
890 while i < num_len and (num_str[i] in string.digits): i = i + 1
900 write_error(":" + line_parts[2], "noweb", int(num_str))
913 \begin_layout Standard
914 Some noweb messages are simply about undefined chunks.
915 These can be seen by looking for matching double-angle-brackets.
920 <<Look for anything with double angle brackets>>=
930 \begin_inset Newline newline
933 left = string.find(line, "<<")
934 \begin_inset Newline newline
937 if (left != -1) and ((left + 2) < len(line)) and
940 \begin_inset Newline newline
943 (string.find(line[left+2:], ">>") != -1):
944 \begin_inset Newline newline
947 write_error(line, "noweb");
948 \begin_inset Newline newline
952 \begin_inset Newline newline
958 \begin_layout Standard
959 Finally, here is an additional list of explicit strings to check for.
964 <<Last ditch effort scan for specific strings>>=
974 msgs_to_try = ("couldn't open file",
979 "couldn't open temporary file",
984 "error writing temporary file",
999 "Bad format sequence",
1004 "Can't open output file",
1009 "Can't open temporary file",
1014 "Capacity exceeded:",
1019 "Ignoring unknown option -",
1024 "This can't happen:",
1029 "non-numeric line number in")
1034 for msg in msgs_to_try:
1039 if string.find(line, msg) != -1:
1044 write_error(line, "noweb")
1062 \begin_layout Subsection
1066 \begin_layout Standard
1067 The gcc errors can be multi-line, with the following format:
1070 \begin_layout LyX-Code
1071 foo.c: In function `main':
1076 foo.c:3: `bar' undeclared (first use in this function)
1081 foo.c:3: (Each undeclared identifier is reported only once
1090 foo.c:3: for each function it appears in.)
1091 \begin_inset Newline newline
1094 foo.c:3: parse error before `x'
1097 \begin_layout Standard
1098 In order to parse this, the gcc error handler has to look ahead and return
1099 any and all lines that do not match the expected pattern.
1104 <<Function Bodies>>=
1114 """See if line is a gcc error.
1115 Read ahead to handle all the lines.
1124 Returns 1 on success, 0 otherwise.
1125 Outputs on stdout."""
1135 <<Handle the gcc error message>>
1152 \begin_layout Standard
1153 The error message starts with a gcc header (as above) without an associated
1159 <<Handle the gcc error message>>=
1160 \begin_inset Newline newline
1163 first_space = string.find(line, ' ')
1164 \begin_inset Newline newline
1167 if first_space > 1: # The smallest would be "X: "
1168 \begin_inset Newline newline
1171 if line[first_space - 1] == ':':
1172 \begin_inset Newline newline
1175 header_to_see = line[:first_space - 1]
1176 \begin_inset Newline newline
1179 next_line = getline()
1180 \begin_inset Newline newline
1183 if next_line and next_line[:first_space - 1] == header_to_see:
1184 \begin_inset Newline newline
1187 num_end = first_space
1188 \begin_inset Newline newline
1191 while next_line[num_end] in string.digits: num_end = num_end + 1
1192 \begin_inset Newline newline
1195 if num_end > first_space: # good!
1196 \begin_inset Newline newline
1199 <<Accumulate gcc error lines and print it>>
1200 \begin_inset Newline newline
1203 else: # oops! Not a gcc error.
1204 \begin_inset Newline newline
1208 \begin_inset Newline newline
1212 \begin_inset Newline newline
1215 pushline(next_line) # return this line to input stream
1216 \begin_inset Newline newline
1222 \begin_layout Standard
1223 At the point in the code that we know that we are in the middle of an error
1224 message, we do the following:
1229 <<Accumulate gcc error lines and print it>>=
1234 num_str = next_line[first_space:num_end]
1239 msgs = [line[first_space:]]
1244 msgs.append(next_line[num_end + 1:])
1249 header_to_see = next_line[:num_end]
1254 next_line = getline()
1259 while next_line and next_line[:num_end] == header_to_see:
1264 msgs.append(next_line[num_end + 1:])
1269 next_line = getline()
1274 if next_line: pushline(next_line)
1279 write_error(msgs, "gcc", int(num_str))
1292 \begin_layout Subsection
1296 \begin_layout Standard
1297 A xlc error message is easy to identify.
1298 Every error message starts with a quoted string with no spaces, a comma,
1300 \begin_inset Quotes eld
1304 \begin_inset Quotes erd
1307 , a space, and some variable text.
1308 The following routine tests if a given buffer line matches this criteria
1309 (this code would probably be simpler if I used the
1310 \begin_inset Quotes eld
1314 \begin_inset Quotes erd
1317 regexp module, but we don't really need the full regular expression engine
1332 <<Function Bodies>>=
1342 """see if line is an xlc error.
1351 Returns 1 on success, 0 otherwise.
1352 Outputs on stdout."""
1362 if line[0] == '"': # This is the first character of all xlc errors
1367 next_quote = string.find(line, '"', 1)
1372 first_space = string.find(line, ' ')
1377 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1383 if line[first_space - 1:first_space + 6] == ", line ":
1388 num_start = num_end = first_space + 6
1393 while line[num_end] in string.digits: num_end = num_end + 1
1398 if num_end > num_start:
1403 write_error(line, "xlc", int(line[num_start : num_end]))
1426 \begin_layout Section
1430 \begin_layout Standard
1431 This project can be tangled from LyX if you set your
1432 \begin_inset Quotes eld
1436 \begin_inset Quotes erd
1439 convertor to call a generic script that always extracts a chunk named
1444 Here is an example of such a generic script:
1447 \begin_layout LyX-Code
1451 \begin_layout LyX-Code
1452 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 NOWEB_OUTPUT_DIR=$2 sh
1455 \begin_layout Standard
1456 This section defines our build-script, which extracts the code.
1471 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1476 if [ -z "$NOWEB_OUTPUT_DIR" ]; then NOWEB_OUTPUT_DIR=.; fi
1481 notangle -Rlisterrors ${NOWEB_SOURCE} > ${NOWEB_OUTPUT_DIR}/listerrors
1486 chmod +x ${NOWEB_OUTPUT_DIR}/listerrors
1494 \begin_layout Section
1498 \begin_layout Standard
1499 This section provides cross-references into the rest of the program.
1502 \begin_layout Subsection
1506 \begin_layout Standard
1510 \begin_layout Plain Layout
1522 \begin_layout Subsection
1526 \begin_layout Standard
1530 \begin_layout Plain Layout