1 #LyX 1.5.0svn created this file. For more info see http://www.lyx.org/
5 \textclass literate-article
10 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
11 \@savsf=1% This is to get around a hyperref+noweb interaction problem
15 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
17 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
18 \let\nwdocspar=\smallbreak
24 \paperfontsize default
31 \paperorientation portrait
34 \paragraph_separation indent
36 \quotes_language english
39 \paperpagestyle default
40 \tracking_changes false
59 \begin_inset LatexCommand \url{mailto:kayvan@sylvan.com}
71 \begin_layout Abstract
73 The listerrors program used to be compiled as a C program and installed
78 along with LyX in order to perform some simple re-formatting of noweb and
80 This document describes and implements the Python version of the same program.
83 \begin_layout Standard
86 \begin_inset LatexCommand \tableofcontents{}
98 \begin_layout Standard
100 The motivation for this program was Bugzilla bug 190
104 \begin_layout Standard
107 \begin_inset LatexCommand \url{http://bugzilla.lyx.org/show_bug.cgi?id=190}
117 \begin_inset Quotes eld
121 \begin_inset Quotes erd
127 \begin_layout Standard
130 \begin_inset Quotes eld
134 \begin_inset Quotes erd
137 ? Usually, LyX has great support for parsing of error messages.
138 For each error in the log file, LyX pops up an error box at that location
140 The error scanning routines expect these errors to be in a certain format
141 (similar to LaTeX errors).
142 When dealing with Literate Programs, you have
143 \begin_inset Quotes eld
150 \begin_layout Standard
153 \begin_inset LatexCommand \url{http://www.eecs.harvard.edu/~nr/noweb}
157 for more information about noweb.
163 \begin_inset Quotes erd
166 as well as gcc error messages (and potentially others).
167 The listerrors program attempts to standardize these error messages to
168 a format that LyX can parse and react to.
171 \begin_layout Standard
173 In a nutshell, the problems with the old implementation of listerrors that
174 bug 190 refers to were::
177 \begin_layout Enumerate
179 It was a C program and it was installed in the user path in the same directory
181 Having such a generically named binary in, for example,
185 , was potentially confusing.
188 \begin_layout Enumerate
190 It required that noweb be installed on the compiling machine (the source
191 was extracted by noweb from
193 SRCDIR/examples/Literate.lyx
195 , compiled and installed by
196 \begin_inset Quotes eld
200 \begin_inset Quotes erd
206 \begin_layout Standard
208 The new version deals with these problems in the following fashion:
211 \begin_layout Enumerate
213 Both the example file (this document) and the program are to be added to
214 the LyX CVS repository.
217 \begin_layout Enumerate
219 The program itself will be installed in
223 , along with other LyX-specific helper scripts.
226 \begin_layout Standard
228 In the design and implementation of this new
229 \begin_inset Quotes eld
233 \begin_inset Quotes erd
240 \begin_layout Standard
242 See the Python home page (
243 \begin_inset LatexCommand \url{http://www.python.org}
247 ) for more information.
252 language was chosen since it is fully multi-platform and provides a very
253 uniform and easy to read syntax.
254 This re-write also simplifies the code for
255 \begin_inset Quotes eld
259 \begin_inset Quotes erd
263 Python is installed by default on all modern Linux systems and is freely
264 available for all other platforms.
271 #!/usr/bin/python -tt
273 """reformat noweb and compiler errors for LyX.
277 Expects to read from stdin and output to stdout.
283 __author__ = "Kayvan A.
284 Sylvan <kayvan@sylvan.com>"
286 __date__ = "$Date: 2005/07/18 09:42:26 $"
288 __version__ = "$Revision: 1.5 $"
290 __credits__ = """Edmar Wienskoski Jr.
291 <edmar-w-jr@technologist.com>
293 original Literate support for LyX.
295 Bernard Michael Hurley <berhardh@westherts.ac.uk>
297 modifications to original listerrors."""
299 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
311 if __name__ == "__main__":
318 \begin_layout Section
320 LaTeX style error message
323 \begin_layout Standard
325 The following function mimics the TeX error message format.
332 def write_error(msg, tool = "noweb", line_number = 1):
334 """Write out the given message in TeX error style.
338 called like: write_error(msg, tool, line_number)."""
340 print "! Build Error: ==> %s ==>
352 if type(msg) == type("str"): # simple string
356 else: # some kind of list (sequence or tuple)
369 \begin_layout Section
374 \begin_layout Standard
376 The only complication in our filtering code is that some parsers might need
377 to push back lines that are read in to be read again later.
378 We solve this problem by implementing a
379 \begin_inset Quotes eld
383 \begin_inset Quotes erd
387 \begin_inset Quotes eld
391 \begin_inset Quotes erd
401 __lines = [] # lines pushed back
405 def getline(file = sys.stdin):
407 """read a line from internal stack or from file.
411 optional file argument defaults to sys.stdin."""
423 line = file.readline()
432 \begin_layout Standard
434 And now for the corresponding pushline function:
443 "push a line onto the pushback stack."
456 \begin_layout Standard
458 The main() entry point function is extremely simple.
459 Note that this version of
460 \begin_inset Quotes eld
464 \begin_inset Quotes erd
467 takes no options and simply filters, attempting simply to match against
468 the known error message patterns.
469 The listerrors C program handled a single-character command-line argument
470 that the current code no longer needs.
480 """Entry point for listerrors.
485 Reads stdin and writes to stdout.
496 <<Check line against patterns and take action>>
501 \begin_layout Standard
503 For each line read in, we need to find out if it matches any of our tools
504 (noweb, gcc, etc.) and act accordingly.
509 <<Check line against patterns and take action>>=
511 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
513 for predicate in try_patterns_dispatch:
515 if predicate(line): break
520 \begin_layout Section
522 Different Error Formats
525 \begin_layout Standard
527 The following sections handle the various error message formats that we
528 recognize in this program.
532 \begin_layout Subsection
537 \begin_layout Standard
539 Noweb errors are output on a single line, so examining just the current
549 """see if line is a noweb error.
553 Returns 1 on success, 0 otherwise.
554 Outputs on stdout."""
558 <<Look for the unescaped angle-brackets in documentation>>
560 <<Look for anything with double angle brackets>>
562 <<Last ditch effort scan for specific strings>>
571 \begin_layout Standard
573 First, we look for the
574 \begin_inset Quotes eld
577 unescaped < < in documentation chunk
578 \begin_inset Quotes erd
582 This is the only message with an associated line number from noweb.
587 <<Look for the unescaped angle-brackets in documentation>>=
589 if string.find(line, ": unescaped << in documentation chunk") != -1:
591 line_parts = string.split(line, ':')
593 num_str = line_parts[1]
595 num_len = len(num_str)
599 while i < num_len and (num_str[i] in string.digits): i = i + 1
603 write_error(":" + line_parts[2], "noweb", int(num_str))
610 \begin_layout Standard
612 Some noweb messages are simply about undefined scraps.
613 These can be seen by looking for matching double-angle-brackets.
618 <<Look for anything with double angle brackets>>=
622 left = string.find(line, "<<")
624 if (left != -1) and ((left + 2) < len(line)) and
628 (string.find(line[left+2:], ">>") != -1):
630 write_error(line, "noweb");
637 \begin_layout Standard
639 Finally, here is an additional list of explicit strings to check for.
644 <<Last ditch effort scan for specific strings>>=
648 msgs_to_try = ("couldn't open file",
650 "couldn't open temporary file",
652 "error writing temporary file",
658 "Bad format sequence",
660 "Can't open output file",
662 "Can't open temporary file",
664 "Capacity exceeded:",
666 "Ignoring unknown option -",
668 "This can't happen:",
670 "non-numeric line number in")
672 for msg in msgs_to_try:
674 if string.find(line, msg) != -1:
676 write_error(line, "noweb")
685 \begin_layout Subsection
690 \begin_layout Standard
692 The gcc errors can be multi-line, with the following format:
695 \begin_layout LyX-Code
697 foo.c: In function `main':
699 foo.c:3: `bar' undeclared (first use in this function)
701 foo.c:3: (Each undeclared identifier is reported only once
703 foo.c:3: for each function it appears in.)
705 foo.c:3: parse error before `x'
708 \begin_layout Standard
710 In order to parse this, the gcc error handler has to look ahead and return
711 any and all lines that do not match the expected pattern.
720 """See if line is a gcc error.
721 Read ahead to handle all the lines.
725 Returns 1 on success, 0 otherwise.
726 Outputs on stdout."""
730 <<Handle the gcc error message>>
739 \begin_layout Standard
741 The error message starts with a gcc header (as above) without an associated
747 <<Handle the gcc error message>>=
749 first_space = string.find(line, ' ')
751 if first_space > 1: # The smallest would be "X: "
753 if line[first_space - 1] == ':':
755 header_to_see = line[:first_space - 1]
757 next_line = getline()
759 if next_line and next_line[:first_space - 1] == header_to_see:
761 num_end = first_space
763 while next_line[num_end] in string.digits: num_end = num_end + 1
765 if num_end > first_space: # good!
767 <<Accumulate gcc error lines and print it>>
769 else: # oops! Not a gcc error.
775 pushline(next_line) # return this line to input stream
780 \begin_layout Standard
782 At the point in the code that we know that we are in the middle of an error
783 message, we do the following:
788 <<Accumulate gcc error lines and print it>>=
790 num_str = next_line[first_space:num_end]
792 msgs = [line[first_space:]]
794 msgs.append(next_line[num_end + 1:])
796 header_to_see = next_line[:num_end]
798 next_line = getline()
800 while next_line and next_line[:num_end] == header_to_see:
802 msgs.append(next_line[num_end + 1:])
804 next_line = getline()
806 if next_line: pushline(next_line)
808 write_error(msgs, "gcc", int(num_str))
815 \begin_layout Subsection
820 \begin_layout Standard
822 A xlc error message is easy to identify.
823 Every error message starts with a quoted string with no spaces, a comma,
825 \begin_inset Quotes eld
829 \begin_inset Quotes erd
832 , a space, and some variable text.
833 The following routine tests if a given buffer line matches this criteria
834 (this code would probably be simpler if I used the
835 \begin_inset Quotes eld
839 \begin_inset Quotes erd
842 regexp module, but we don't really need the full regular expression engine
853 """see if line is an xlc error.
857 Returns 1 on success, 0 otherwise.
858 Outputs on stdout."""
862 if line[0] == '"': # This is the first character of all xlc errors
864 next_quote = string.find(line, '"', 1)
866 first_space = string.find(line, ' ')
868 if (next_quote != -1) and (first_space > next_quote): # no space inisde
871 if line[first_space - 1:first_space + 6] == ", line ":
873 num_start = num_end = first_space + 6
875 while line[num_end] in string.digits: num_end = num_end + 1
877 if num_end > num_start:
879 write_error(line, "xlc", int(line[num_start : num_end]))
890 \begin_layout Section
895 \begin_layout Standard
897 This project can be tangled from LyX if you set your
898 \begin_inset Quotes eld
902 \begin_inset Quotes erd
905 convertor to call a generic script that always extracts a scrap named
910 Here is an example of such a generic script:
913 \begin_layout LyX-Code
917 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 sh
920 \begin_layout Standard
922 This section defines our build-script, which extracts the code.
931 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
933 notangle -Rlisterrors ${NOWEB_SOURCE} > listerrors
940 \begin_layout Section
945 \begin_layout Standard
947 This section provides cross-references into the rest of the program.
950 \begin_layout Subsection
955 \begin_layout Standard
961 \begin_layout Standard
972 \begin_layout Subsection
977 \begin_layout Standard
983 \begin_layout Standard