1 #LyX 1.6.0 created this file. For more info see http://www.lyx.org/
5 \use_default_options false
6 \textclass literate-article
11 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
12 \@savsf=1% This is to get around a hyperref+noweb interaction problem
16 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
18 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
19 \let\nwdocspar=\smallbreak
25 \font_typewriter courier
26 \font_default_family default
32 \paperfontsize default
41 \paperorientation portrait
44 \paragraph_separation indent
46 \quotes_language english
49 \paperpagestyle default
50 \tracking_changes false
59 \begin_inset Newline newline
68 \begin_inset Newline newline
75 \begin_layout Plain Layout
77 mailto:kayvan@sylvan.com
89 \begin_layout Abstract
90 The listerrors program used to be compiled as a C program and installed
95 along with LyX in order to perform some simple re-formatting of noweb and
97 This document describes and implements the Python version of the same program.
100 \begin_layout Standard
101 \begin_inset CommandInset toc
102 LatexCommand tableofcontents
109 \begin_layout Section
113 \begin_layout Standard
114 The motivation for this program was Bugzilla bug 190
118 \begin_layout Plain Layout
120 \begin_inset Flex URL
123 \begin_layout Plain Layout
125 http://bugzilla.lyx.org/show_bug.cgi?id=190
136 \begin_inset Quotes eld
140 \begin_inset Quotes erd
146 \begin_layout Standard
148 \begin_inset Quotes eld
152 \begin_inset Quotes erd
155 ? Usually, LyX has great support for parsing of error messages.
156 For each error in the log file, LyX pops up an error box at that location
158 The error scanning routines expect these errors to be in a certain format
159 (similar to LaTeX errors).
160 When dealing with Literate Programs, you have
161 \begin_inset Quotes eld
168 \begin_layout Plain Layout
170 \begin_inset Flex URL
173 \begin_layout Plain Layout
175 http://www.eecs.harvard.edu/~nr/noweb
180 for more information about noweb.
186 \begin_inset Quotes erd
189 as well as gcc error messages (and potentially others).
190 The listerrors program attempts to standardize these error messages to
191 a format that LyX can parse and react to.
194 \begin_layout Standard
195 In a nutshell, the problems with the old implementation of listerrors that
196 bug 190 refers to were::
199 \begin_layout Enumerate
200 It was a C program and it was installed in the user path in the same directory
202 Having such a generically named binary in, for example,
206 , was potentially confusing.
209 \begin_layout Enumerate
210 It required that noweb be installed on the compiling machine (the source
211 was extracted by noweb from
213 SRCDIR/examples/Literate.lyx
215 , compiled and installed by
216 \begin_inset Quotes eld
220 \begin_inset Quotes erd
226 \begin_layout Standard
227 The new version deals with these problems in the following fashion:
230 \begin_layout Enumerate
231 Both the example file (this document) and the program are to be added to
232 the LyX CVS repository.
235 \begin_layout Enumerate
236 The program itself will be installed in
240 , along with other LyX-specific helper scripts.
243 \begin_layout Standard
244 In the design and implementation of this new
245 \begin_inset Quotes eld
249 \begin_inset Quotes erd
256 \begin_layout Plain Layout
257 See the Python home page (
258 \begin_inset Flex URL
261 \begin_layout Plain Layout
263 http://www.python.org
268 ) for more information.
273 language was chosen since it is fully multi-platform and provides a very
274 uniform and easy to read syntax.
275 This re-write also simplifies the code for
276 \begin_inset Quotes eld
280 \begin_inset Quotes erd
284 Python is installed by default on all modern Linux systems and is freely
285 available for all other platforms.
290 \begin_inset Newline newline
293 #!/usr/bin/python -tt
294 \begin_inset Newline newline
297 """reformat noweb and compiler errors for LyX.
298 \begin_inset Newline newline
302 \begin_inset Newline newline
305 Expects to read from stdin and output to stdout.
306 \begin_inset Newline newline
310 \begin_inset Newline newline
314 \begin_inset Newline newline
317 __author__ = "Kayvan A.
318 Sylvan <kayvan@sylvan.com>"
319 \begin_inset Newline newline
322 __date__ = "$Date: 2005/07/18 09:42:26 $"
323 \begin_inset Newline newline
326 __version__ = "$Revision: 1.5 $"
327 \begin_inset Newline newline
330 __credits__ = """Edmar Wienskoski Jr.
331 <edmar-w-jr@technologist.com>
332 \begin_inset Newline newline
335 original Literate support for LyX.
336 \begin_inset Newline newline
339 Bernard Michael Hurley <berhardh@westherts.ac.uk>
340 \begin_inset Newline newline
343 modifications to original listerrors."""
344 \begin_inset Newline newline
347 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
348 \begin_inset Newline newline
352 \begin_inset Newline newline
356 \begin_inset Newline newline
360 \begin_inset Newline newline
364 \begin_inset Newline newline
368 \begin_inset Newline newline
371 if __name__ == "__main__":
372 \begin_inset Newline newline
376 \begin_inset Newline newline
382 \begin_layout Section
383 LaTeX style error message
386 \begin_layout Standard
387 The following function mimics the TeX error message format.
392 \begin_inset Newline newline
395 def write_error(msg, tool = "noweb", line_number = 1):
396 \begin_inset Newline newline
399 """Write out the given message in TeX error style.
400 \begin_inset Newline newline
404 \begin_inset Newline newline
407 called like: write_error(msg, tool, line_number)."""
408 \begin_inset Newline newline
411 print "! Build Error: ==> %s ==>
414 \begin_inset Newline newline
424 \begin_inset Newline newline
427 if type(msg) == type("str"): # simple string
428 \begin_inset Newline newline
432 \begin_inset Newline newline
435 else: # some kind of list (sequence or tuple)
436 \begin_inset Newline newline
440 \begin_inset Newline newline
444 \begin_inset Newline newline
448 \begin_inset Newline newline
452 \begin_inset Newline newline
458 \begin_layout Section
462 \begin_layout Standard
463 The only complication in our filtering code is that some parsers might need
464 to push back lines that are read in to be read again later.
465 We solve this problem by implementing a
466 \begin_inset Quotes eld
470 \begin_inset Quotes erd
474 \begin_inset Quotes eld
478 \begin_inset Quotes erd
486 \begin_inset Newline newline
489 __lines = [] # lines pushed back
490 \begin_inset Newline newline
494 \begin_inset Newline newline
497 def getline(file = sys.stdin):
498 \begin_inset Newline newline
501 """read a line from internal stack or from file.
502 \begin_inset Newline newline
506 \begin_inset Newline newline
509 optional file argument defaults to sys.stdin."""
510 \begin_inset Newline newline
514 \begin_inset Newline newline
518 \begin_inset Newline newline
522 \begin_inset Newline newline
526 \begin_inset Newline newline
530 \begin_inset Newline newline
533 line = file.readline()
534 \begin_inset Newline newline
538 \begin_inset Newline newline
542 \begin_inset Newline newline
548 \begin_layout Standard
549 And now for the corresponding pushline function:
554 \begin_inset Newline newline
558 \begin_inset Newline newline
561 "push a line onto the pushback stack."
562 \begin_inset Newline newline
566 \begin_inset Newline newline
570 \begin_inset Newline newline
574 \begin_inset Newline newline
578 \begin_inset Newline newline
584 \begin_layout Standard
585 The main() entry point function is extremely simple.
586 Note that this version of
587 \begin_inset Quotes eld
591 \begin_inset Quotes erd
594 takes no options and simply filters, attempting simply to match against
595 the known error message patterns.
596 The listerrors C program handled a single-character command-line argument
597 that the current code no longer needs.
603 \begin_inset Newline newline
607 \begin_inset Newline newline
610 """Entry point for listerrors.
612 \begin_inset Newline newline
616 \begin_inset Newline newline
619 Reads stdin and writes to stdout.
621 \begin_inset Newline newline
625 \begin_inset Newline newline
629 \begin_inset Newline newline
633 \begin_inset Newline newline
637 \begin_inset Newline newline
640 <<Check line against patterns and take action>>
641 \begin_inset Newline newline
647 \begin_layout Standard
648 For each line read in, we need to find out if it matches any of our tools
649 (noweb, gcc, etc.) and act accordingly.
653 <<Check line against patterns and take action>>=
654 \begin_inset Newline newline
657 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
658 \begin_inset Newline newline
661 for predicate in try_patterns_dispatch:
662 \begin_inset Newline newline
665 if predicate(line): break
666 \begin_inset Newline newline
672 \begin_layout Section
673 Different Error Formats
676 \begin_layout Standard
677 The following sections handle the various error message formats that we
678 recognize in this program.
682 \begin_layout Subsection
686 \begin_layout Standard
687 Noweb errors are output on a single line, so examining just the current
693 \begin_inset Newline newline
697 \begin_inset Newline newline
700 """see if line is a noweb error.
701 \begin_inset Newline newline
705 \begin_inset Newline newline
708 Returns 1 on success, 0 otherwise.
709 Outputs on stdout."""
710 \begin_inset Newline newline
714 \begin_inset Newline newline
717 <<Look for the unescaped angle-brackets in documentation>>
718 \begin_inset Newline newline
721 <<Look for anything with double angle brackets>>
722 \begin_inset Newline newline
725 <<Last ditch effort scan for specific strings>>
726 \begin_inset Newline newline
730 \begin_inset Newline newline
734 \begin_inset Newline newline
740 \begin_layout Standard
741 First, we look for the
742 \begin_inset Quotes eld
745 unescaped < < in documentation chunk
746 \begin_inset Quotes erd
750 This is the only message with an associated line number from noweb.
754 <<Look for the unescaped angle-brackets in documentation>>=
755 \begin_inset Newline newline
758 if string.find(line, ": unescaped << in documentation chunk") != -1:
759 \begin_inset Newline newline
762 line_parts = string.split(line, ':')
763 \begin_inset Newline newline
766 num_str = line_parts[1]
767 \begin_inset Newline newline
770 num_len = len(num_str)
771 \begin_inset Newline newline
775 \begin_inset Newline newline
778 while i < num_len and (num_str[i] in string.digits): i = i + 1
779 \begin_inset Newline newline
783 \begin_inset Newline newline
786 write_error(":" + line_parts[2], "noweb", int(num_str))
787 \begin_inset Newline newline
791 \begin_inset Newline newline
797 \begin_layout Standard
798 Some noweb messages are simply about undefined scraps.
799 These can be seen by looking for matching double-angle-brackets.
803 <<Look for anything with double angle brackets>>=
804 \begin_inset Newline newline
808 \begin_inset Newline newline
811 left = string.find(line, "<<")
812 \begin_inset Newline newline
815 if (left != -1) and ((left + 2) < len(line)) and
818 \begin_inset Newline newline
821 (string.find(line[left+2:], ">>") != -1):
822 \begin_inset Newline newline
825 write_error(line, "noweb");
826 \begin_inset Newline newline
830 \begin_inset Newline newline
836 \begin_layout Standard
837 Finally, here is an additional list of explicit strings to check for.
841 <<Last ditch effort scan for specific strings>>=
842 \begin_inset Newline newline
846 \begin_inset Newline newline
849 msgs_to_try = ("couldn't open file",
850 \begin_inset Newline newline
853 "couldn't open temporary file",
854 \begin_inset Newline newline
857 "error writing temporary file",
858 \begin_inset Newline newline
862 \begin_inset Newline newline
866 \begin_inset Newline newline
869 "Bad format sequence",
870 \begin_inset Newline newline
873 "Can't open output file",
874 \begin_inset Newline newline
877 "Can't open temporary file",
878 \begin_inset Newline newline
881 "Capacity exceeded:",
882 \begin_inset Newline newline
885 "Ignoring unknown option -",
886 \begin_inset Newline newline
889 "This can't happen:",
890 \begin_inset Newline newline
893 "non-numeric line number in")
894 \begin_inset Newline newline
897 for msg in msgs_to_try:
898 \begin_inset Newline newline
901 if string.find(line, msg) != -1:
902 \begin_inset Newline newline
905 write_error(line, "noweb")
906 \begin_inset Newline newline
910 \begin_inset Newline newline
914 \begin_inset Newline newline
920 \begin_layout Subsection
924 \begin_layout Standard
925 The gcc errors can be multi-line, with the following format:
928 \begin_layout LyX-Code
929 foo.c: In function `main':
930 \begin_inset Newline newline
933 foo.c:3: `bar' undeclared (first use in this function)
934 \begin_inset Newline newline
937 foo.c:3: (Each undeclared identifier is reported only once
938 \begin_inset Newline newline
941 foo.c:3: for each function it appears in.)
942 \begin_inset Newline newline
945 foo.c:3: parse error before `x'
948 \begin_layout Standard
949 In order to parse this, the gcc error handler has to look ahead and return
950 any and all lines that do not match the expected pattern.
955 \begin_inset Newline newline
959 \begin_inset Newline newline
962 """See if line is a gcc error.
963 Read ahead to handle all the lines.
964 \begin_inset Newline newline
968 \begin_inset Newline newline
971 Returns 1 on success, 0 otherwise.
972 Outputs on stdout."""
973 \begin_inset Newline newline
977 \begin_inset Newline newline
980 <<Handle the gcc error message>>
981 \begin_inset Newline newline
985 \begin_inset Newline newline
989 \begin_inset Newline newline
995 \begin_layout Standard
996 The error message starts with a gcc header (as above) without an associated
1001 <<Handle the gcc error message>>=
1002 \begin_inset Newline newline
1005 first_space = string.find(line, ' ')
1006 \begin_inset Newline newline
1009 if first_space > 1: # The smallest would be "X: "
1010 \begin_inset Newline newline
1013 if line[first_space - 1] == ':':
1014 \begin_inset Newline newline
1017 header_to_see = line[:first_space - 1]
1018 \begin_inset Newline newline
1021 next_line = getline()
1022 \begin_inset Newline newline
1025 if next_line and next_line[:first_space - 1] == header_to_see:
1026 \begin_inset Newline newline
1029 num_end = first_space
1030 \begin_inset Newline newline
1033 while next_line[num_end] in string.digits: num_end = num_end + 1
1034 \begin_inset Newline newline
1037 if num_end > first_space: # good!
1038 \begin_inset Newline newline
1041 <<Accumulate gcc error lines and print it>>
1042 \begin_inset Newline newline
1045 else: # oops! Not a gcc error.
1046 \begin_inset Newline newline
1050 \begin_inset Newline newline
1054 \begin_inset Newline newline
1057 pushline(next_line) # return this line to input stream
1058 \begin_inset Newline newline
1064 \begin_layout Standard
1065 At the point in the code that we know that we are in the middle of an error
1066 message, we do the following:
1070 <<Accumulate gcc error lines and print it>>=
1071 \begin_inset Newline newline
1074 num_str = next_line[first_space:num_end]
1075 \begin_inset Newline newline
1078 msgs = [line[first_space:]]
1079 \begin_inset Newline newline
1082 msgs.append(next_line[num_end + 1:])
1083 \begin_inset Newline newline
1086 header_to_see = next_line[:num_end]
1087 \begin_inset Newline newline
1090 next_line = getline()
1091 \begin_inset Newline newline
1094 while next_line and next_line[:num_end] == header_to_see:
1095 \begin_inset Newline newline
1098 msgs.append(next_line[num_end + 1:])
1099 \begin_inset Newline newline
1102 next_line = getline()
1103 \begin_inset Newline newline
1106 if next_line: pushline(next_line)
1107 \begin_inset Newline newline
1110 write_error(msgs, "gcc", int(num_str))
1111 \begin_inset Newline newline
1115 \begin_inset Newline newline
1121 \begin_layout Subsection
1125 \begin_layout Standard
1126 A xlc error message is easy to identify.
1127 Every error message starts with a quoted string with no spaces, a comma,
1129 \begin_inset Quotes eld
1133 \begin_inset Quotes erd
1136 , a space, and some variable text.
1137 The following routine tests if a given buffer line matches this criteria
1138 (this code would probably be simpler if I used the
1139 \begin_inset Quotes eld
1143 \begin_inset Quotes erd
1146 regexp module, but we don't really need the full regular expression engine
1152 <<Function Bodies>>=
1153 \begin_inset Newline newline
1157 \begin_inset Newline newline
1160 """see if line is an xlc error.
1161 \begin_inset Newline newline
1165 \begin_inset Newline newline
1168 Returns 1 on success, 0 otherwise.
1169 Outputs on stdout."""
1170 \begin_inset Newline newline
1174 \begin_inset Newline newline
1177 if line[0] == '"': # This is the first character of all xlc errors
1178 \begin_inset Newline newline
1181 next_quote = string.find(line, '"', 1)
1182 \begin_inset Newline newline
1185 first_space = string.find(line, ' ')
1186 \begin_inset Newline newline
1189 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1191 \begin_inset Newline newline
1194 if line[first_space - 1:first_space + 6] == ", line ":
1195 \begin_inset Newline newline
1198 num_start = num_end = first_space + 6
1199 \begin_inset Newline newline
1202 while line[num_end] in string.digits: num_end = num_end + 1
1203 \begin_inset Newline newline
1206 if num_end > num_start:
1207 \begin_inset Newline newline
1210 write_error(line, "xlc", int(line[num_start : num_end]))
1211 \begin_inset Newline newline
1215 \begin_inset Newline newline
1219 \begin_inset Newline newline
1223 \begin_inset Newline newline
1229 \begin_layout Section
1233 \begin_layout Standard
1234 This project can be tangled from LyX if you set your
1235 \begin_inset Quotes eld
1239 \begin_inset Quotes erd
1242 convertor to call a generic script that always extracts a scrap named
1247 Here is an example of such a generic script:
1250 \begin_layout LyX-Code
1252 \begin_inset Newline newline
1255 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 sh
1258 \begin_layout Standard
1259 This section defines our build-script, which extracts the code.
1264 \begin_inset Newline newline
1268 \begin_inset Newline newline
1271 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1272 \begin_inset Newline newline
1275 notangle -Rlisterrors ${NOWEB_SOURCE} > listerrors
1276 \begin_inset Newline newline
1280 \begin_inset Newline newline
1286 \begin_layout Section
1290 \begin_layout Standard
1291 This section provides cross-references into the rest of the program.
1294 \begin_layout Subsection
1298 \begin_layout Standard
1302 \begin_layout Plain Layout
1314 \begin_layout Subsection
1318 \begin_layout Standard
1322 \begin_layout Plain Layout