1 #LyX 2.1 created this file. For more info see http://www.lyx.org/
5 \textclass literate-article
10 \usepackage[ps2pdf,pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
11 \@savsf=1% This is to get around a hyperref+noweb interaction problem
15 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
17 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
18 \let\nwdocspar=\smallbreak
20 \use_default_options false
21 \maintain_unincluded_children false
23 \language_package default
28 \font_typewriter courier
30 \font_default_family default
31 \use_non_tex_fonts false
37 \default_output_format default
39 \bibtex_command default
40 \index_command default
41 \paperfontsize default
46 \use_package amsmath 0
47 \use_package amssymb 0
49 \use_package mathdots 1
50 \use_package mathtools 0
52 \use_package undertilde 0
54 \cite_engine_type numerical
58 \paperorientation portrait
68 \paragraph_separation indent
69 \paragraph_indentation default
70 \quotes_language english
73 \paperpagestyle default
74 \tracking_changes false
85 \begin_inset Newline newline
94 \begin_inset Newline newline
101 \begin_layout Plain Layout
103 mailto:kayvan@sylvan.com
115 \begin_layout Abstract
116 The listerrors program used to be compiled as a C program and installed
121 along with LyX in order to perform some simple re-formatting of noweb and
123 This document describes and implements the Python version of the same program.
126 \begin_layout Standard
127 \begin_inset CommandInset toc
128 LatexCommand tableofcontents
135 \begin_layout Section
139 \begin_layout Standard
140 The motivation for this program was LyX bug 190
144 \begin_layout Plain Layout
146 \begin_inset Flex URL
149 \begin_layout Plain Layout
151 http://www.lyx.org/trac/ticket/190
162 \begin_inset Quotes eld
166 \begin_inset Quotes erd
172 \begin_layout Standard
174 \begin_inset Quotes eld
178 \begin_inset Quotes erd
181 ? Usually, LyX has great support for parsing of error messages.
182 For each error in the log file, LyX pops up an error box at that location
184 The error scanning routines expect these errors to be in a certain format
185 (similar to LaTeX errors).
186 When dealing with Literate Programs, you have
187 \begin_inset Quotes eld
194 \begin_layout Plain Layout
196 \begin_inset Flex URL
199 \begin_layout Plain Layout
201 http://www.eecs.harvard.edu/~nr/noweb
206 for more information about noweb.
212 \begin_inset Quotes erd
215 as well as gcc error messages (and potentially others).
216 The listerrors program attempts to standardize these error messages to
217 a format that LyX can parse and react to.
220 \begin_layout Standard
221 In a nutshell, the problems with the old implementation of listerrors that
222 bug 190 refers to were::
225 \begin_layout Enumerate
226 It was a C program and it was installed in the user path in the same directory
228 Having such a generically named binary in, for example,
232 , was potentially confusing.
235 \begin_layout Enumerate
236 It required that noweb be installed on the compiling machine (the source
237 was extracted by noweb from
239 SRCDIR/examples/Literate.lyx
241 , compiled and installed by
242 \begin_inset Quotes eld
246 \begin_inset Quotes erd
252 \begin_layout Standard
253 The new version deals with these problems in the following fashion:
256 \begin_layout Enumerate
257 Both the example file (this document) and the program are to be added to
258 the LyX CVS repository.
261 \begin_layout Enumerate
262 The program itself will be installed in
266 , along with other LyX-specific helper scripts.
269 \begin_layout Standard
270 In the design and implementation of this new
271 \begin_inset Quotes eld
275 \begin_inset Quotes erd
282 \begin_layout Plain Layout
283 See the Python home page (
284 \begin_inset Flex URL
287 \begin_layout Plain Layout
289 http://www.python.org
294 ) for more information.
299 language was chosen since it is fully multi-platform and provides a very
300 uniform and easy to read syntax.
301 This re-write also simplifies the code for
302 \begin_inset Quotes eld
306 \begin_inset Quotes erd
310 Python is installed by default on all modern Linux systems and is freely
311 available for all other platforms.
321 #!/usr/bin/python -tt
326 """reformat noweb and compiler errors for LyX.
335 Expects to read from stdin and output to stdout.
349 __author__ = "Kayvan A.
350 Sylvan <kayvan@sylvan.com>"
355 __date__ = "$Date: 2005/07/18 09:42:26 $"
360 __version__ = "$Revision: 1.5 $"
365 __credits__ = """Edmar Wienskoski Jr.
366 <edmar-w-jr@technologist.com>
371 original Literate support for LyX.
376 Bernard Michael Hurley <berhardh@westherts.ac.uk>
381 modifications to original listerrors."""
386 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
413 if __name__ == "__main__":
426 \begin_layout Section
427 LaTeX style error message
430 \begin_layout Standard
431 The following function mimics the TeX error message format.
441 def write_error(msg, tool = "noweb", line_number = 1):
446 """Write out the given message in TeX error style.
459 called like: write_error(msg, tool, line_number)."""
460 \begin_inset Newline newline
463 print "! Build Error: ==> %s ==>
466 \begin_inset Newline newline
476 \begin_inset Newline newline
479 if type(msg) == type("str"): # simple string
480 \begin_inset Newline newline
484 \begin_inset Newline newline
487 else: # some kind of list (sequence or tuple)
488 \begin_inset Newline newline
492 \begin_inset Newline newline
496 \begin_inset Newline newline
500 \begin_inset Newline newline
504 \begin_inset Newline newline
510 \begin_layout Section
514 \begin_layout Standard
515 The only complication in our filtering code is that some parsers might need
516 to push back lines that are read in to be read again later.
517 We solve this problem by implementing a
518 \begin_inset Quotes eld
522 \begin_inset Quotes erd
526 \begin_inset Quotes eld
530 \begin_inset Quotes erd
543 __lines = [] # lines pushed back
552 def getline(file = sys.stdin):
557 """read a line from internal stack or from file.
566 optional file argument defaults to sys.stdin."""
596 line = file.readline()
613 \begin_layout Standard
614 And now for the corresponding pushline function:
629 "push a line onto the pushback stack."
639 \begin_inset Newline newline
643 \begin_inset Newline newline
647 \begin_inset Newline newline
651 \begin_inset Newline newline
657 \begin_layout Standard
658 The main() entry point function is extremely simple.
659 Note that this version of
660 \begin_inset Quotes eld
664 \begin_inset Quotes erd
667 takes no options and simply filters, attempting simply to match against
668 the known error message patterns.
669 The listerrors C program handled a single-character command-line argument
670 that the current code no longer needs.
686 """Entry point for listerrors.
696 Reads stdin and writes to stdout.
721 <<Check line against patterns and take action>>
729 \begin_layout Standard
730 For each line read in, we need to find out if it matches any of our tools
731 (noweb, gcc, etc.) and act accordingly.
736 <<Check line against patterns and take action>>=
745 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
746 \begin_inset Newline newline
749 for predicate in try_patterns_dispatch:
750 \begin_inset Newline newline
753 if predicate(line): break
754 \begin_inset Newline newline
760 \begin_layout Section
761 Different Error Formats
764 \begin_layout Standard
765 The following sections handle the various error message formats that we
766 recognize in this program.
770 \begin_layout Subsection
774 \begin_layout Standard
775 Noweb errors are output on a single line, so examining just the current
791 """see if line is a noweb error.
800 Returns 1 on success, 0 otherwise.
801 Outputs on stdout."""
811 <<Look for the unescaped angle-brackets in documentation>>
816 <<Look for anything with double angle brackets>>
821 <<Last ditch effort scan for specific strings>>
838 \begin_layout Standard
839 First, we look for the
840 \begin_inset Quotes eld
843 unescaped < < in documentation chunk
844 \begin_inset Quotes erd
848 This is the only message with an associated line number from noweb.
857 <<Look for the unescaped angle-brackets in documentation>>=
862 if string.find(line, ": unescaped << in documentation chunk") != -1:
867 line_parts = string.split(line, ':')
872 num_str = line_parts[1]
877 num_len = len(num_str)
887 while i < num_len and (num_str[i] in string.digits): i = i + 1
897 write_error(":" + line_parts[2], "noweb", int(num_str))
910 \begin_layout Standard
911 Some noweb messages are simply about undefined scraps.
912 These can be seen by looking for matching double-angle-brackets.
917 <<Look for anything with double angle brackets>>=
927 \begin_inset Newline newline
930 left = string.find(line, "<<")
931 \begin_inset Newline newline
934 if (left != -1) and ((left + 2) < len(line)) and
937 \begin_inset Newline newline
940 (string.find(line[left+2:], ">>") != -1):
941 \begin_inset Newline newline
944 write_error(line, "noweb");
945 \begin_inset Newline newline
949 \begin_inset Newline newline
955 \begin_layout Standard
956 Finally, here is an additional list of explicit strings to check for.
961 <<Last ditch effort scan for specific strings>>=
971 msgs_to_try = ("couldn't open file",
976 "couldn't open temporary file",
981 "error writing temporary file",
996 "Bad format sequence",
1001 "Can't open output file",
1006 "Can't open temporary file",
1011 "Capacity exceeded:",
1016 "Ignoring unknown option -",
1021 "This can't happen:",
1026 "non-numeric line number in")
1031 for msg in msgs_to_try:
1036 if string.find(line, msg) != -1:
1041 write_error(line, "noweb")
1059 \begin_layout Subsection
1063 \begin_layout Standard
1064 The gcc errors can be multi-line, with the following format:
1067 \begin_layout LyX-Code
1068 foo.c: In function `main':
1073 foo.c:3: `bar' undeclared (first use in this function)
1078 foo.c:3: (Each undeclared identifier is reported only once
1087 foo.c:3: for each function it appears in.)
1088 \begin_inset Newline newline
1091 foo.c:3: parse error before `x'
1094 \begin_layout Standard
1095 In order to parse this, the gcc error handler has to look ahead and return
1096 any and all lines that do not match the expected pattern.
1101 <<Function Bodies>>=
1111 """See if line is a gcc error.
1112 Read ahead to handle all the lines.
1121 Returns 1 on success, 0 otherwise.
1122 Outputs on stdout."""
1132 <<Handle the gcc error message>>
1149 \begin_layout Standard
1150 The error message starts with a gcc header (as above) without an associated
1156 <<Handle the gcc error message>>=
1157 \begin_inset Newline newline
1160 first_space = string.find(line, ' ')
1161 \begin_inset Newline newline
1164 if first_space > 1: # The smallest would be "X: "
1165 \begin_inset Newline newline
1168 if line[first_space - 1] == ':':
1169 \begin_inset Newline newline
1172 header_to_see = line[:first_space - 1]
1173 \begin_inset Newline newline
1176 next_line = getline()
1177 \begin_inset Newline newline
1180 if next_line and next_line[:first_space - 1] == header_to_see:
1181 \begin_inset Newline newline
1184 num_end = first_space
1185 \begin_inset Newline newline
1188 while next_line[num_end] in string.digits: num_end = num_end + 1
1189 \begin_inset Newline newline
1192 if num_end > first_space: # good!
1193 \begin_inset Newline newline
1196 <<Accumulate gcc error lines and print it>>
1197 \begin_inset Newline newline
1200 else: # oops! Not a gcc error.
1201 \begin_inset Newline newline
1205 \begin_inset Newline newline
1209 \begin_inset Newline newline
1212 pushline(next_line) # return this line to input stream
1213 \begin_inset Newline newline
1219 \begin_layout Standard
1220 At the point in the code that we know that we are in the middle of an error
1221 message, we do the following:
1226 <<Accumulate gcc error lines and print it>>=
1231 num_str = next_line[first_space:num_end]
1236 msgs = [line[first_space:]]
1241 msgs.append(next_line[num_end + 1:])
1246 header_to_see = next_line[:num_end]
1251 next_line = getline()
1256 while next_line and next_line[:num_end] == header_to_see:
1261 msgs.append(next_line[num_end + 1:])
1266 next_line = getline()
1271 if next_line: pushline(next_line)
1276 write_error(msgs, "gcc", int(num_str))
1289 \begin_layout Subsection
1293 \begin_layout Standard
1294 A xlc error message is easy to identify.
1295 Every error message starts with a quoted string with no spaces, a comma,
1297 \begin_inset Quotes eld
1301 \begin_inset Quotes erd
1304 , a space, and some variable text.
1305 The following routine tests if a given buffer line matches this criteria
1306 (this code would probably be simpler if I used the
1307 \begin_inset Quotes eld
1311 \begin_inset Quotes erd
1314 regexp module, but we don't really need the full regular expression engine
1329 <<Function Bodies>>=
1339 """see if line is an xlc error.
1348 Returns 1 on success, 0 otherwise.
1349 Outputs on stdout."""
1359 if line[0] == '"': # This is the first character of all xlc errors
1364 next_quote = string.find(line, '"', 1)
1369 first_space = string.find(line, ' ')
1374 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1380 if line[first_space - 1:first_space + 6] == ", line ":
1385 num_start = num_end = first_space + 6
1390 while line[num_end] in string.digits: num_end = num_end + 1
1395 if num_end > num_start:
1400 write_error(line, "xlc", int(line[num_start : num_end]))
1423 \begin_layout Section
1427 \begin_layout Standard
1428 This project can be tangled from LyX if you set your
1429 \begin_inset Quotes eld
1433 \begin_inset Quotes erd
1436 convertor to call a generic script that always extracts a scrap named
1441 Here is an example of such a generic script:
1444 \begin_layout LyX-Code
1446 \begin_inset Newline newline
1449 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 sh
1452 \begin_layout Standard
1453 This section defines our build-script, which extracts the code.
1468 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1473 if [ -z "$NOWEB_OUTPUT_DIR" ]; then NOWEB_OUTPUT_DIR=.; fi
1478 notangle -Rlisterrors ${NOWEB_SOURCE} > ${NOWEB_OUTPUT_DIR}/listerrors
1483 chmod +x ${NOWEB_OUTPUT_DIR}/listerrors
1491 \begin_layout Section
1495 \begin_layout Standard
1496 This section provides cross-references into the rest of the program.
1499 \begin_layout Subsection
1503 \begin_layout Standard
1507 \begin_layout Plain Layout
1519 \begin_layout Subsection
1523 \begin_layout Standard
1527 \begin_layout Plain Layout
projects / lyx.git / blob