1 #LyX 1.5.0svn created this file. For more info see http://www.lyx.org/
5 \textclass literate-article
10 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
11 \@savsf=1% This is to get around a hyperref+noweb interaction problem
15 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
17 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
18 \let\nwdocspar=\smallbreak
24 \font_typewriter courier
25 \font_default_family default
31 \paperfontsize default
39 \paperorientation portrait
42 \paragraph_separation indent
44 \quotes_language english
47 \paperpagestyle default
48 \tracking_changes false
67 \begin_inset LatexCommand url
68 target "mailto:kayvan@sylvan.com"
79 \begin_layout Abstract
81 The listerrors program used to be compiled as a C program and installed
88 along with LyX in order to perform some simple re-formatting of noweb and
90 This document describes and implements the Python version of the same program.
93 \begin_layout Standard
96 \begin_inset LatexCommand tableofcontents
102 \begin_layout Section
107 \begin_layout Standard
109 The motivation for this program was Bugzilla bug 190
113 \begin_layout Standard
116 \begin_inset LatexCommand url
117 target "http://bugzilla.lyx.org/show_bug.cgi?id=190"
126 \begin_inset Quotes eld
130 \begin_inset Quotes erd
136 \begin_layout Standard
139 \begin_inset Quotes eld
143 \begin_inset Quotes erd
146 ? Usually, LyX has great support for parsing of error messages.
147 For each error in the log file, LyX pops up an error box at that location
149 The error scanning routines expect these errors to be in a certain format
150 (similar to LaTeX errors).
151 When dealing with Literate Programs, you have
152 \begin_inset Quotes eld
159 \begin_layout Standard
162 \begin_inset LatexCommand url
163 target "http://www.eecs.harvard.edu/~nr/noweb"
166 for more information about noweb.
172 \begin_inset Quotes erd
175 as well as gcc error messages (and potentially others).
176 The listerrors program attempts to standardize these error messages to
177 a format that LyX can parse and react to.
180 \begin_layout Standard
182 In a nutshell, the problems with the old implementation of listerrors that
183 bug 190 refers to were::
186 \begin_layout Enumerate
188 It was a C program and it was installed in the user path in the same directory
190 Having such a generically named binary in, for example,
196 , was potentially confusing.
199 \begin_layout Enumerate
201 It required that noweb be installed on the compiling machine (the source
202 was extracted by noweb from
206 SRCDIR/examples/Literate.lyx
208 , compiled and installed by
209 \begin_inset Quotes eld
213 \begin_inset Quotes erd
219 \begin_layout Standard
221 The new version deals with these problems in the following fashion:
224 \begin_layout Enumerate
226 Both the example file (this document) and the program are to be added to
227 the LyX CVS repository.
230 \begin_layout Enumerate
232 The program itself will be installed in
238 , along with other LyX-specific helper scripts.
241 \begin_layout Standard
243 In the design and implementation of this new
244 \begin_inset Quotes eld
248 \begin_inset Quotes erd
255 \begin_layout Standard
257 See the Python home page (
258 \begin_inset LatexCommand url
259 target "http://www.python.org"
262 ) for more information.
267 language was chosen since it is fully multi-platform and provides a very
268 uniform and easy to read syntax.
269 This re-write also simplifies the code for
270 \begin_inset Quotes eld
274 \begin_inset Quotes erd
278 Python is installed by default on all modern Linux systems and is freely
279 available for all other platforms.
286 #!/usr/bin/python -tt
288 """reformat noweb and compiler errors for LyX.
292 Expects to read from stdin and output to stdout.
298 __author__ = "Kayvan A.
299 Sylvan <kayvan@sylvan.com>"
301 __date__ = "$Date: 2005/07/18 09:42:26 $"
303 __version__ = "$Revision: 1.5 $"
305 __credits__ = """Edmar Wienskoski Jr.
306 <edmar-w-jr@technologist.com>
308 original Literate support for LyX.
310 Bernard Michael Hurley <berhardh@westherts.ac.uk>
312 modifications to original listerrors."""
314 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
326 if __name__ == "__main__":
333 \begin_layout Section
335 LaTeX style error message
338 \begin_layout Standard
340 The following function mimics the TeX error message format.
347 def write_error(msg, tool = "noweb", line_number = 1):
349 """Write out the given message in TeX error style.
353 called like: write_error(msg, tool, line_number)."""
355 print "! Build Error: ==> %s ==>
367 if type(msg) == type("str"): # simple string
371 else: # some kind of list (sequence or tuple)
384 \begin_layout Section
389 \begin_layout Standard
391 The only complication in our filtering code is that some parsers might need
392 to push back lines that are read in to be read again later.
393 We solve this problem by implementing a
394 \begin_inset Quotes eld
398 \begin_inset Quotes erd
402 \begin_inset Quotes eld
406 \begin_inset Quotes erd
416 __lines = [] # lines pushed back
420 def getline(file = sys.stdin):
422 """read a line from internal stack or from file.
426 optional file argument defaults to sys.stdin."""
438 line = file.readline()
447 \begin_layout Standard
449 And now for the corresponding pushline function:
458 "push a line onto the pushback stack."
471 \begin_layout Standard
473 The main() entry point function is extremely simple.
474 Note that this version of
475 \begin_inset Quotes eld
479 \begin_inset Quotes erd
482 takes no options and simply filters, attempting simply to match against
483 the known error message patterns.
484 The listerrors C program handled a single-character command-line argument
485 that the current code no longer needs.
495 """Entry point for listerrors.
500 Reads stdin and writes to stdout.
511 <<Check line against patterns and take action>>
516 \begin_layout Standard
518 For each line read in, we need to find out if it matches any of our tools
519 (noweb, gcc, etc.) and act accordingly.
524 <<Check line against patterns and take action>>=
526 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
528 for predicate in try_patterns_dispatch:
530 if predicate(line): break
535 \begin_layout Section
537 Different Error Formats
540 \begin_layout Standard
542 The following sections handle the various error message formats that we
543 recognize in this program.
547 \begin_layout Subsection
552 \begin_layout Standard
554 Noweb errors are output on a single line, so examining just the current
564 """see if line is a noweb error.
568 Returns 1 on success, 0 otherwise.
569 Outputs on stdout."""
573 <<Look for the unescaped angle-brackets in documentation>>
575 <<Look for anything with double angle brackets>>
577 <<Last ditch effort scan for specific strings>>
586 \begin_layout Standard
588 First, we look for the
589 \begin_inset Quotes eld
592 unescaped < < in documentation chunk
593 \begin_inset Quotes erd
597 This is the only message with an associated line number from noweb.
602 <<Look for the unescaped angle-brackets in documentation>>=
604 if string.find(line, ": unescaped << in documentation chunk") != -1:
606 line_parts = string.split(line, ':')
608 num_str = line_parts[1]
610 num_len = len(num_str)
614 while i < num_len and (num_str[i] in string.digits): i = i + 1
618 write_error(":" + line_parts[2], "noweb", int(num_str))
625 \begin_layout Standard
627 Some noweb messages are simply about undefined scraps.
628 These can be seen by looking for matching double-angle-brackets.
633 <<Look for anything with double angle brackets>>=
637 left = string.find(line, "<<")
639 if (left != -1) and ((left + 2) < len(line)) and
643 (string.find(line[left+2:], ">>") != -1):
645 write_error(line, "noweb");
652 \begin_layout Standard
654 Finally, here is an additional list of explicit strings to check for.
659 <<Last ditch effort scan for specific strings>>=
663 msgs_to_try = ("couldn't open file",
665 "couldn't open temporary file",
667 "error writing temporary file",
673 "Bad format sequence",
675 "Can't open output file",
677 "Can't open temporary file",
679 "Capacity exceeded:",
681 "Ignoring unknown option -",
683 "This can't happen:",
685 "non-numeric line number in")
687 for msg in msgs_to_try:
689 if string.find(line, msg) != -1:
691 write_error(line, "noweb")
700 \begin_layout Subsection
705 \begin_layout Standard
707 The gcc errors can be multi-line, with the following format:
710 \begin_layout LyX-Code
712 foo.c: In function `main':
714 foo.c:3: `bar' undeclared (first use in this function)
716 foo.c:3: (Each undeclared identifier is reported only once
718 foo.c:3: for each function it appears in.)
720 foo.c:3: parse error before `x'
723 \begin_layout Standard
725 In order to parse this, the gcc error handler has to look ahead and return
726 any and all lines that do not match the expected pattern.
735 """See if line is a gcc error.
736 Read ahead to handle all the lines.
740 Returns 1 on success, 0 otherwise.
741 Outputs on stdout."""
745 <<Handle the gcc error message>>
754 \begin_layout Standard
756 The error message starts with a gcc header (as above) without an associated
762 <<Handle the gcc error message>>=
764 first_space = string.find(line, ' ')
766 if first_space > 1: # The smallest would be "X: "
768 if line[first_space - 1] == ':':
770 header_to_see = line[:first_space - 1]
772 next_line = getline()
774 if next_line and next_line[:first_space - 1] == header_to_see:
776 num_end = first_space
778 while next_line[num_end] in string.digits: num_end = num_end + 1
780 if num_end > first_space: # good!
782 <<Accumulate gcc error lines and print it>>
784 else: # oops! Not a gcc error.
790 pushline(next_line) # return this line to input stream
795 \begin_layout Standard
797 At the point in the code that we know that we are in the middle of an error
798 message, we do the following:
803 <<Accumulate gcc error lines and print it>>=
805 num_str = next_line[first_space:num_end]
807 msgs = [line[first_space:]]
809 msgs.append(next_line[num_end + 1:])
811 header_to_see = next_line[:num_end]
813 next_line = getline()
815 while next_line and next_line[:num_end] == header_to_see:
817 msgs.append(next_line[num_end + 1:])
819 next_line = getline()
821 if next_line: pushline(next_line)
823 write_error(msgs, "gcc", int(num_str))
830 \begin_layout Subsection
835 \begin_layout Standard
837 A xlc error message is easy to identify.
838 Every error message starts with a quoted string with no spaces, a comma,
840 \begin_inset Quotes eld
844 \begin_inset Quotes erd
847 , a space, and some variable text.
848 The following routine tests if a given buffer line matches this criteria
849 (this code would probably be simpler if I used the
850 \begin_inset Quotes eld
854 \begin_inset Quotes erd
857 regexp module, but we don't really need the full regular expression engine
868 """see if line is an xlc error.
872 Returns 1 on success, 0 otherwise.
873 Outputs on stdout."""
877 if line[0] == '"': # This is the first character of all xlc errors
879 next_quote = string.find(line, '"', 1)
881 first_space = string.find(line, ' ')
883 if (next_quote != -1) and (first_space > next_quote): # no space inisde
886 if line[first_space - 1:first_space + 6] == ", line ":
888 num_start = num_end = first_space + 6
890 while line[num_end] in string.digits: num_end = num_end + 1
892 if num_end > num_start:
894 write_error(line, "xlc", int(line[num_start : num_end]))
905 \begin_layout Section
910 \begin_layout Standard
912 This project can be tangled from LyX if you set your
913 \begin_inset Quotes eld
917 \begin_inset Quotes erd
920 convertor to call a generic script that always extracts a scrap named
927 Here is an example of such a generic script:
930 \begin_layout LyX-Code
934 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 sh
937 \begin_layout Standard
939 This section defines our build-script, which extracts the code.
948 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
950 notangle -Rlisterrors ${NOWEB_SOURCE} > listerrors
957 \begin_layout Section
962 \begin_layout Standard
964 This section provides cross-references into the rest of the program.
967 \begin_layout Subsection
972 \begin_layout Standard
978 \begin_layout Standard
989 \begin_layout Subsection
994 \begin_layout Standard
1000 \begin_layout Standard