1 #LyX 2.0 created this file. For more info see http://www.lyx.org/
5 \textclass literate-article
10 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
11 \@savsf=1% This is to get around a hyperref+noweb interaction problem
15 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
17 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
18 \let\nwdocspar=\smallbreak
20 \use_default_options false
21 \maintain_unincluded_children false
23 \language_package default
28 \font_typewriter courier
29 \font_default_family default
30 \use_non_tex_fonts false
37 \default_output_format default
39 \bibtex_command default
40 \index_command default
41 \paperfontsize default
53 \paperorientation portrait
62 \paragraph_separation indent
63 \paragraph_indentation default
64 \quotes_language english
67 \paperpagestyle default
68 \tracking_changes false
79 \begin_inset Newline newline
88 \begin_inset Newline newline
95 \begin_layout Plain Layout
97 mailto:kayvan@sylvan.com
109 \begin_layout Abstract
110 The listerrors program used to be compiled as a C program and installed
115 along with LyX in order to perform some simple re-formatting of noweb and
117 This document describes and implements the Python version of the same program.
120 \begin_layout Standard
121 \begin_inset CommandInset toc
122 LatexCommand tableofcontents
129 \begin_layout Section
133 \begin_layout Standard
134 The motivation for this program was Bugzilla bug 190
138 \begin_layout Plain Layout
140 \begin_inset Flex URL
143 \begin_layout Plain Layout
145 http://bugzilla.lyx.org/show_bug.cgi?id=190
156 \begin_inset Quotes eld
160 \begin_inset Quotes erd
166 \begin_layout Standard
168 \begin_inset Quotes eld
172 \begin_inset Quotes erd
175 ? Usually, LyX has great support for parsing of error messages.
176 For each error in the log file, LyX pops up an error box at that location
178 The error scanning routines expect these errors to be in a certain format
179 (similar to LaTeX errors).
180 When dealing with Literate Programs, you have
181 \begin_inset Quotes eld
188 \begin_layout Plain Layout
190 \begin_inset Flex URL
193 \begin_layout Plain Layout
195 http://www.eecs.harvard.edu/~nr/noweb
200 for more information about noweb.
206 \begin_inset Quotes erd
209 as well as gcc error messages (and potentially others).
210 The listerrors program attempts to standardize these error messages to
211 a format that LyX can parse and react to.
214 \begin_layout Standard
215 In a nutshell, the problems with the old implementation of listerrors that
216 bug 190 refers to were::
219 \begin_layout Enumerate
220 It was a C program and it was installed in the user path in the same directory
222 Having such a generically named binary in, for example,
226 , was potentially confusing.
229 \begin_layout Enumerate
230 It required that noweb be installed on the compiling machine (the source
231 was extracted by noweb from
233 SRCDIR/examples/Literate.lyx
235 , compiled and installed by
236 \begin_inset Quotes eld
240 \begin_inset Quotes erd
246 \begin_layout Standard
247 The new version deals with these problems in the following fashion:
250 \begin_layout Enumerate
251 Both the example file (this document) and the program are to be added to
252 the LyX CVS repository.
255 \begin_layout Enumerate
256 The program itself will be installed in
260 , along with other LyX-specific helper scripts.
263 \begin_layout Standard
264 In the design and implementation of this new
265 \begin_inset Quotes eld
269 \begin_inset Quotes erd
276 \begin_layout Plain Layout
277 See the Python home page (
278 \begin_inset Flex URL
281 \begin_layout Plain Layout
283 http://www.python.org
288 ) for more information.
293 language was chosen since it is fully multi-platform and provides a very
294 uniform and easy to read syntax.
295 This re-write also simplifies the code for
296 \begin_inset Quotes eld
300 \begin_inset Quotes erd
304 Python is installed by default on all modern Linux systems and is freely
305 available for all other platforms.
313 #!/usr/bin/python -tt
317 """reformat noweb and compiler errors for LyX.
325 Expects to read from stdin and output to stdout.
337 __author__ = "Kayvan A.
338 Sylvan <kayvan@sylvan.com>"
342 __date__ = "$Date: 2005/07/18 09:42:26 $"
346 __version__ = "$Revision: 1.5 $"
350 __credits__ = """Edmar Wienskoski Jr.
351 <edmar-w-jr@technologist.com>
355 original Literate support for LyX.
359 Bernard Michael Hurley <berhardh@westherts.ac.uk>
363 modifications to original listerrors."""
367 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
391 if __name__ == "__main__":
402 \begin_layout Section
403 LaTeX style error message
406 \begin_layout Standard
407 The following function mimics the TeX error message format.
415 def write_error(msg, tool = "noweb", line_number = 1):
419 """Write out the given message in TeX error style.
431 called like: write_error(msg, tool, line_number)."""
432 \begin_inset Newline newline
435 print "! Build Error: ==> %s ==>
438 \begin_inset Newline newline
448 \begin_inset Newline newline
451 if type(msg) == type("str"): # simple string
452 \begin_inset Newline newline
456 \begin_inset Newline newline
459 else: # some kind of list (sequence or tuple)
460 \begin_inset Newline newline
464 \begin_inset Newline newline
468 \begin_inset Newline newline
472 \begin_inset Newline newline
476 \begin_inset Newline newline
482 \begin_layout Section
486 \begin_layout Standard
487 The only complication in our filtering code is that some parsers might need
488 to push back lines that are read in to be read again later.
489 We solve this problem by implementing a
490 \begin_inset Quotes eld
494 \begin_inset Quotes erd
498 \begin_inset Quotes eld
502 \begin_inset Quotes erd
513 __lines = [] # lines pushed back
521 def getline(file = sys.stdin):
525 """read a line from internal stack or from file.
533 optional file argument defaults to sys.stdin."""
557 line = file.readline()
572 \begin_layout Standard
573 And now for the corresponding pushline function:
585 "push a line onto the pushback stack."
594 \begin_inset Newline newline
598 \begin_inset Newline newline
602 \begin_inset Newline newline
606 \begin_inset Newline newline
612 \begin_layout Standard
613 The main() entry point function is extremely simple.
614 Note that this version of
615 \begin_inset Quotes eld
619 \begin_inset Quotes erd
622 takes no options and simply filters, attempting simply to match against
623 the known error message patterns.
624 The listerrors C program handled a single-character command-line argument
625 that the current code no longer needs.
638 """Entry point for listerrors.
647 Reads stdin and writes to stdout.
668 <<Check line against patterns and take action>>
675 \begin_layout Standard
676 For each line read in, we need to find out if it matches any of our tools
677 (noweb, gcc, etc.) and act accordingly.
681 <<Check line against patterns and take action>>=
689 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
690 \begin_inset Newline newline
693 for predicate in try_patterns_dispatch:
694 \begin_inset Newline newline
697 if predicate(line): break
698 \begin_inset Newline newline
704 \begin_layout Section
705 Different Error Formats
708 \begin_layout Standard
709 The following sections handle the various error message formats that we
710 recognize in this program.
714 \begin_layout Subsection
718 \begin_layout Standard
719 Noweb errors are output on a single line, so examining just the current
732 """see if line is a noweb error.
740 Returns 1 on success, 0 otherwise.
741 Outputs on stdout."""
749 <<Look for the unescaped angle-brackets in documentation>>
753 <<Look for anything with double angle brackets>>
757 <<Last ditch effort scan for specific strings>>
772 \begin_layout Standard
773 First, we look for the
774 \begin_inset Quotes eld
777 unescaped < < in documentation chunk
778 \begin_inset Quotes erd
782 This is the only message with an associated line number from noweb.
790 <<Look for the unescaped angle-brackets in documentation>>=
794 if string.find(line, ": unescaped << in documentation chunk") != -1:
798 line_parts = string.split(line, ':')
802 num_str = line_parts[1]
806 num_len = len(num_str)
814 while i < num_len and (num_str[i] in string.digits): i = i + 1
822 write_error(":" + line_parts[2], "noweb", int(num_str))
833 \begin_layout Standard
834 Some noweb messages are simply about undefined scraps.
835 These can be seen by looking for matching double-angle-brackets.
839 <<Look for anything with double angle brackets>>=
848 \begin_inset Newline newline
851 left = string.find(line, "<<")
852 \begin_inset Newline newline
855 if (left != -1) and ((left + 2) < len(line)) and
858 \begin_inset Newline newline
861 (string.find(line[left+2:], ">>") != -1):
862 \begin_inset Newline newline
865 write_error(line, "noweb");
866 \begin_inset Newline newline
870 \begin_inset Newline newline
876 \begin_layout Standard
877 Finally, here is an additional list of explicit strings to check for.
881 <<Last ditch effort scan for specific strings>>=
889 msgs_to_try = ("couldn't open file",
893 "couldn't open temporary file",
897 "error writing temporary file",
909 "Bad format sequence",
913 "Can't open output file",
917 "Can't open temporary file",
921 "Capacity exceeded:",
925 "Ignoring unknown option -",
929 "This can't happen:",
933 "non-numeric line number in")
937 for msg in msgs_to_try:
941 if string.find(line, msg) != -1:
945 write_error(line, "noweb")
960 \begin_layout Subsection
964 \begin_layout Standard
965 The gcc errors can be multi-line, with the following format:
968 \begin_layout LyX-Code
969 foo.c: In function `main':
973 foo.c:3: `bar' undeclared (first use in this function)
977 foo.c:3: (Each undeclared identifier is reported only once
985 foo.c:3: for each function it appears in.)
986 \begin_inset Newline newline
989 foo.c:3: parse error before `x'
992 \begin_layout Standard
993 In order to parse this, the gcc error handler has to look ahead and return
994 any and all lines that do not match the expected pattern.
1006 """See if line is a gcc error.
1007 Read ahead to handle all the lines.
1015 Returns 1 on success, 0 otherwise.
1016 Outputs on stdout."""
1024 <<Handle the gcc error message>>
1039 \begin_layout Standard
1040 The error message starts with a gcc header (as above) without an associated
1045 <<Handle the gcc error message>>=
1046 \begin_inset Newline newline
1049 first_space = string.find(line, ' ')
1050 \begin_inset Newline newline
1053 if first_space > 1: # The smallest would be "X: "
1054 \begin_inset Newline newline
1057 if line[first_space - 1] == ':':
1058 \begin_inset Newline newline
1061 header_to_see = line[:first_space - 1]
1062 \begin_inset Newline newline
1065 next_line = getline()
1066 \begin_inset Newline newline
1069 if next_line and next_line[:first_space - 1] == header_to_see:
1070 \begin_inset Newline newline
1073 num_end = first_space
1074 \begin_inset Newline newline
1077 while next_line[num_end] in string.digits: num_end = num_end + 1
1078 \begin_inset Newline newline
1081 if num_end > first_space: # good!
1082 \begin_inset Newline newline
1085 <<Accumulate gcc error lines and print it>>
1086 \begin_inset Newline newline
1089 else: # oops! Not a gcc error.
1090 \begin_inset Newline newline
1094 \begin_inset Newline newline
1098 \begin_inset Newline newline
1101 pushline(next_line) # return this line to input stream
1102 \begin_inset Newline newline
1108 \begin_layout Standard
1109 At the point in the code that we know that we are in the middle of an error
1110 message, we do the following:
1114 <<Accumulate gcc error lines and print it>>=
1118 num_str = next_line[first_space:num_end]
1122 msgs = [line[first_space:]]
1126 msgs.append(next_line[num_end + 1:])
1130 header_to_see = next_line[:num_end]
1134 next_line = getline()
1138 while next_line and next_line[:num_end] == header_to_see:
1142 msgs.append(next_line[num_end + 1:])
1146 next_line = getline()
1150 if next_line: pushline(next_line)
1154 write_error(msgs, "gcc", int(num_str))
1165 \begin_layout Subsection
1169 \begin_layout Standard
1170 A xlc error message is easy to identify.
1171 Every error message starts with a quoted string with no spaces, a comma,
1173 \begin_inset Quotes eld
1177 \begin_inset Quotes erd
1180 , a space, and some variable text.
1181 The following routine tests if a given buffer line matches this criteria
1182 (this code would probably be simpler if I used the
1183 \begin_inset Quotes eld
1187 \begin_inset Quotes erd
1190 regexp module, but we don't really need the full regular expression engine
1204 <<Function Bodies>>=
1212 """see if line is an xlc error.
1220 Returns 1 on success, 0 otherwise.
1221 Outputs on stdout."""
1229 if line[0] == '"': # This is the first character of all xlc errors
1233 next_quote = string.find(line, '"', 1)
1237 first_space = string.find(line, ' ')
1241 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1246 if line[first_space - 1:first_space + 6] == ", line ":
1250 num_start = num_end = first_space + 6
1254 while line[num_end] in string.digits: num_end = num_end + 1
1258 if num_end > num_start:
1262 write_error(line, "xlc", int(line[num_start : num_end]))
1281 \begin_layout Section
1285 \begin_layout Standard
1286 This project can be tangled from LyX if you set your
1287 \begin_inset Quotes eld
1291 \begin_inset Quotes erd
1294 convertor to call a generic script that always extracts a scrap named
1299 Here is an example of such a generic script:
1302 \begin_layout LyX-Code
1304 \begin_inset Newline newline
1307 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 sh
1310 \begin_layout Standard
1311 This section defines our build-script, which extracts the code.
1323 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1327 notangle -Rlisterrors ${NOWEB_SOURCE} > listerrors
1338 \begin_layout Section
1342 \begin_layout Standard
1343 This section provides cross-references into the rest of the program.
1346 \begin_layout Subsection
1350 \begin_layout Standard
1354 \begin_layout Plain Layout
1366 \begin_layout Subsection
1370 \begin_layout Standard
1374 \begin_layout Plain Layout
projects / lyx.git / blob