1 #LyX 2.4 created this file. For more info see https://www.lyx.org/
5 \save_transient_properties true
6 \origin /systemlyxdir/examples/Modules/
9 \usepackage[pdftitle={LyX listerrors re-implemented},urlcolor=blue,linktocpage,letterpaper,colorlinks=true]{hyperref}
10 \@savsf=1% This is to get around a hyperref+noweb interaction problem
14 % This (from the noweb FAQ) relaxes the constraint that chunks are never broken across pages.
16 \def\nwendcode{\endtrivlist \endgroup \vfil\penalty10\vfilneg}
17 \let\nwdocspar=\smallbreak
19 \use_default_options false
23 \maintain_unincluded_children no
25 \language_package default
28 \font_roman "times" "default"
29 \font_sans "helvet" "default"
30 \font_typewriter "courier" "default"
31 \font_math "auto" "auto"
32 \font_default_family default
33 \use_non_tex_fonts false
36 \font_typewriter_osf false
38 \font_sf_scale 100 100
39 \font_tt_scale 100 100
41 \use_dash_ligatures false
43 \default_output_format default
45 \bibtex_command default
46 \index_command default
47 \paperfontsize default
52 \use_package amsmath 1
53 \use_package amssymb 1
56 \use_package mathdots 1
57 \use_package mathtools 1
59 \use_package stackrel 1
60 \use_package stmaryrd 1
61 \use_package undertilde 1
63 \cite_engine_type default
67 \paperorientation portrait
79 \paragraph_separation indent
80 \paragraph_indentation default
82 \math_numbering_side default
87 \paperpagestyle default
88 \tracking_changes false
89 \postpone_fragile_content false
101 \begin_inset Newline newline
110 \begin_inset Newline newline
114 \begin_inset Flex URL
117 \begin_layout Plain Layout
119 mailto:kayvan@sylvan.com
131 \begin_layout Abstract
132 The listerrors program used to be compiled as a C program and installed
137 along with \SpecialChar LyX
138 in order to perform some simple re-formatting of noweb and
140 This document describes and implements the Python version of the same program.
143 \begin_layout Standard
144 \begin_inset CommandInset toc
145 LatexCommand tableofcontents
152 \begin_layout Section
156 \begin_layout Standard
157 The motivation for this program was \SpecialChar LyX
162 \begin_layout Plain Layout
164 \begin_inset Flex URL
167 \begin_layout Plain Layout
169 https://www.lyx.org/trac/ticket/190
180 \begin_inset Quotes eld
184 \begin_inset Quotes erd
190 \begin_layout Standard
192 \begin_inset Quotes eld
196 \begin_inset Quotes erd
199 ? Usually, \SpecialChar LyX
200 has great support for parsing of error messages.
201 For each error in the log file, \SpecialChar LyX
202 pops up an error box at that location
203 in the \SpecialChar LyX
205 The error scanning routines expect these errors to be in a certain format
206 (similar to \SpecialChar LaTeX
208 When dealing with Literate Programs, you have
209 \begin_inset Quotes eld
216 \begin_layout Plain Layout
218 \begin_inset Flex URL
221 \begin_layout Plain Layout
223 https://www.cs.tufts.edu/~nr/noweb/
228 for more information about noweb.
234 \begin_inset Quotes erd
237 as well as gcc error messages (and potentially others).
238 The listerrors program attempts to standardize these error messages to
239 a format that \SpecialChar LyX
240 can parse and react to.
243 \begin_layout Standard
244 In a nutshell, the problems with the old implementation of listerrors that
245 bug 190 refers to were::
248 \begin_layout Enumerate
249 It was a C program and it was installed in the user path in the same directory
252 Having such a generically named binary in, for example,
256 , was potentially confusing.
259 \begin_layout Enumerate
260 It required that noweb be installed on the compiling machine (the source
261 was extracted by noweb from
263 SRCDIR/examples/Literate.lyx
265 , compiled and installed by
266 \begin_inset Quotes eld
270 \begin_inset Quotes erd
276 \begin_layout Standard
277 The new version deals with these problems in the following fashion:
280 \begin_layout Enumerate
281 Both the example file (this document) and the program are to be added to
286 \begin_layout Enumerate
287 The program itself will be installed in
291 , along with other \SpecialChar LyX
292 -specific helper scripts.
295 \begin_layout Standard
296 In the design and implementation of this new
297 \begin_inset Quotes eld
301 \begin_inset Quotes erd
308 \begin_layout Plain Layout
309 See the Python home page (
310 \begin_inset Flex URL
313 \begin_layout Plain Layout
315 http://www.python.org
320 ) for more information.
325 language was chosen since it is fully multi-platform and provides a very
326 uniform and easy to read syntax.
327 This re-write also simplifies the code for
328 \begin_inset Quotes eld
332 \begin_inset Quotes erd
336 Python is installed by default on all modern Linux systems and is freely
337 available for all other platforms.
340 \begin_layout Standard
344 \begin_layout Plain Layout
349 \begin_layout Plain Layout
351 #!/usr/bin/python -tt
354 \begin_layout Plain Layout
356 """reformat noweb and compiler errors for LyX.
359 \begin_layout Plain Layout
363 \begin_layout Plain Layout
365 Expects to read from stdin and output to stdout.
368 \begin_layout Plain Layout
373 \begin_layout Plain Layout
377 \begin_layout Plain Layout
379 __author__ = "Kayvan A.
380 Sylvan <kayvan@sylvan.com>"
383 \begin_layout Plain Layout
385 __date__ = "$Date: 2005/07/18 09:42:26 $"
388 \begin_layout Plain Layout
390 __version__ = "$Revision: 1.5 $"
393 \begin_layout Plain Layout
395 __credits__ = """Edmar Wienskoski Jr.
396 <edmar-w-jr@technologist.com>
399 \begin_layout Plain Layout
401 original Literate support for LyX.
404 \begin_layout Plain Layout
406 Bernard Michael Hurley <berhardh@westherts.ac.uk>
409 \begin_layout Plain Layout
411 modifications to original listerrors."""
414 \begin_layout Plain Layout
416 __copyright__ = "Copyright 2002 - Kayvan Sylvan."
419 \begin_layout Plain Layout
423 \begin_layout Plain Layout
428 \begin_layout Plain Layout
432 \begin_layout Plain Layout
437 \begin_layout Plain Layout
441 \begin_layout Plain Layout
443 if __name__ == "__main__":
446 \begin_layout Plain Layout
451 \begin_layout Plain Layout
461 \begin_layout Section
466 \begin_layout Standard
467 The following function mimics the \SpecialChar TeX
468 error message format.
471 \begin_layout Standard
475 \begin_layout Plain Layout
480 \begin_layout Plain Layout
482 def write_error(msg, tool = "noweb", line_number = 1):
485 \begin_layout Plain Layout
487 """Write out the given message in TeX error style.
490 \begin_layout Plain Layout
494 \begin_layout Plain Layout
498 \begin_layout Plain Layout
500 called like: write_error(msg, tool, line_number)."""
503 \begin_layout Plain Layout
505 print "! Build Error: ==> %s ==>
510 \begin_layout Plain Layout
521 \begin_layout Plain Layout
523 if type(msg) == type("str"): # simple string
526 \begin_layout Plain Layout
531 \begin_layout Plain Layout
533 else: # some kind of list (sequence or tuple)
536 \begin_layout Plain Layout
541 \begin_layout Plain Layout
546 \begin_layout Plain Layout
551 \begin_layout Plain Layout
555 \begin_layout Plain Layout
565 \begin_layout Section
569 \begin_layout Standard
570 The only complication in our filtering code is that some parsers might need
571 to push back lines that are read in to be read again later.
572 We solve this problem by implementing a
573 \begin_inset Quotes eld
577 \begin_inset Quotes erd
581 \begin_inset Quotes eld
585 \begin_inset Quotes erd
591 \begin_layout Standard
595 \begin_layout Plain Layout
600 \begin_layout Plain Layout
602 __lines = [] # lines pushed back
605 \begin_layout Plain Layout
609 \begin_layout Plain Layout
611 def getline(file = sys.stdin):
614 \begin_layout Plain Layout
616 """read a line from internal stack or from file.
619 \begin_layout Plain Layout
623 \begin_layout Plain Layout
625 optional file argument defaults to sys.stdin."""
628 \begin_layout Plain Layout
633 \begin_layout Plain Layout
638 \begin_layout Plain Layout
643 \begin_layout Plain Layout
648 \begin_layout Plain Layout
653 \begin_layout Plain Layout
655 line = file.readline()
658 \begin_layout Plain Layout
663 \begin_layout Plain Layout
667 \begin_layout Plain Layout
677 \begin_layout Standard
678 And now for the corresponding pushline function:
681 \begin_layout Standard
685 \begin_layout Plain Layout
690 \begin_layout Plain Layout
695 \begin_layout Plain Layout
697 "push a line onto the pushback stack."
700 \begin_layout Plain Layout
704 \begin_layout Plain Layout
709 \begin_layout Plain Layout
714 \begin_layout Plain Layout
719 \begin_layout Plain Layout
723 \begin_layout Plain Layout
733 \begin_layout Standard
734 The main() entry point function is extremely simple.
735 Note that this version of
736 \begin_inset Quotes eld
740 \begin_inset Quotes erd
743 takes no options and simply filters, attempting simply to match against
744 the known error message patterns.
745 The listerrors C program handled a single-character command-line argument
746 that the current code no longer needs.
750 \begin_layout Standard
754 \begin_layout Plain Layout
759 \begin_layout Plain Layout
764 \begin_layout Plain Layout
766 """Entry point for listerrors.
770 \begin_layout Plain Layout
774 \begin_layout Plain Layout
776 Reads stdin and writes to stdout.
780 \begin_layout Plain Layout
784 \begin_layout Plain Layout
789 \begin_layout Plain Layout
794 \begin_layout Plain Layout
799 \begin_layout Plain Layout
801 <<Check line against patterns and take action>>
804 \begin_layout Plain Layout
814 \begin_layout Standard
815 For each line read in, we need to find out if it matches any of our tools
816 (noweb, gcc, etc.) and act accordingly.
819 \begin_layout Standard
823 \begin_layout Plain Layout
825 <<Check line against patterns and take action>>=
828 \begin_layout Plain Layout
832 \begin_layout Plain Layout
834 try_patterns_dispatch = [ noweb_try, gcc_try, xlc_try ]
837 \begin_layout Plain Layout
839 for predicate in try_patterns_dispatch:
842 \begin_layout Plain Layout
844 if predicate(line): break
847 \begin_layout Plain Layout
857 \begin_layout Section
858 Different Error Formats
861 \begin_layout Standard
862 The following sections handle the various error message formats that we
863 recognize in this program.
867 \begin_layout Subsection
871 \begin_layout Standard
872 Noweb errors are output on a single line, so examining just the current
876 \begin_layout Standard
880 \begin_layout Plain Layout
885 \begin_layout Plain Layout
890 \begin_layout Plain Layout
892 """see if line is a noweb error.
895 \begin_layout Plain Layout
899 \begin_layout Plain Layout
901 Returns 1 on success, 0 otherwise.
902 Outputs on stdout."""
905 \begin_layout Plain Layout
910 \begin_layout Plain Layout
912 <<Look for the unescaped angle-brackets in documentation>>
915 \begin_layout Plain Layout
917 <<Look for anything with double angle brackets>>
920 \begin_layout Plain Layout
922 <<Last ditch effort scan for specific strings>>
925 \begin_layout Plain Layout
930 \begin_layout Plain Layout
934 \begin_layout Plain Layout
944 \begin_layout Standard
945 First, we look for the
946 \begin_inset Quotes eld
949 unescaped < < in documentation chunk
950 \begin_inset Quotes erd
954 This is the only message with an associated line number from noweb.
957 \begin_layout Standard
961 \begin_layout Plain Layout
965 \begin_layout Plain Layout
967 <<Look for the unescaped angle-brackets in documentation>>=
970 \begin_layout Plain Layout
972 if string.find(line, ": unescaped << in documentation chunk") != -1:
975 \begin_layout Plain Layout
977 line_parts = string.split(line, ':')
980 \begin_layout Plain Layout
982 num_str = line_parts[1]
985 \begin_layout Plain Layout
987 num_len = len(num_str)
990 \begin_layout Plain Layout
995 \begin_layout Plain Layout
997 while i < num_len and (num_str[i] in string.digits): i = i + 1
1000 \begin_layout Plain Layout
1005 \begin_layout Plain Layout
1007 write_error(":" + line_parts[2], "noweb", int(num_str))
1010 \begin_layout Plain Layout
1015 \begin_layout Plain Layout
1025 \begin_layout Standard
1026 Some noweb messages are simply about undefined chunks.
1027 These can be seen by looking for matching double-angle-brackets.
1030 \begin_layout Standard
1034 \begin_layout Plain Layout
1036 <<Look for anything with double angle brackets>>=
1039 \begin_layout Plain Layout
1043 \begin_layout Plain Layout
1048 \begin_layout Plain Layout
1050 left = string.find(line, "<<")
1053 \begin_layout Plain Layout
1055 if (left != -1) and ((left + 2) < len(line)) and
1060 \begin_layout Plain Layout
1062 (string.find(line[left+2:], ">>") != -1):
1065 \begin_layout Plain Layout
1067 write_error(line, "noweb");
1070 \begin_layout Plain Layout
1075 \begin_layout Plain Layout
1085 \begin_layout Standard
1086 Finally, here is an additional list of explicit strings to check for.
1089 \begin_layout Standard
1093 \begin_layout Plain Layout
1095 <<Last ditch effort scan for specific strings>>=
1098 \begin_layout Plain Layout
1103 \begin_layout Plain Layout
1105 msgs_to_try = ("couldn't open file",
1108 \begin_layout Plain Layout
1110 "couldn't open temporary file",
1113 \begin_layout Plain Layout
1115 "error writing temporary file",
1118 \begin_layout Plain Layout
1120 "ill-formed option",
1123 \begin_layout Plain Layout
1128 \begin_layout Plain Layout
1130 "Bad format sequence",
1133 \begin_layout Plain Layout
1135 "Can't open output file",
1138 \begin_layout Plain Layout
1140 "Can't open temporary file",
1143 \begin_layout Plain Layout
1145 "Capacity exceeded:",
1148 \begin_layout Plain Layout
1150 "Ignoring unknown option -",
1153 \begin_layout Plain Layout
1155 "This can't happen:",
1158 \begin_layout Plain Layout
1160 "non-numeric line number in")
1163 \begin_layout Plain Layout
1165 for msg in msgs_to_try:
1168 \begin_layout Plain Layout
1170 if string.find(line, msg) != -1:
1173 \begin_layout Plain Layout
1175 write_error(line, "noweb")
1178 \begin_layout Plain Layout
1183 \begin_layout Plain Layout
1188 \begin_layout Plain Layout
1198 \begin_layout Subsection
1202 \begin_layout Standard
1203 The gcc errors can be multi-line, with the following format:
1206 \begin_layout LyX-Code
1207 foo.c: In function `main':
1210 \begin_layout Standard
1214 \begin_layout Plain Layout
1216 foo.c:3: `bar' undeclared (first use in this function)
1219 \begin_layout Plain Layout
1221 foo.c:3: (Each undeclared identifier is reported only once
1224 \begin_layout Plain Layout
1228 \begin_layout Plain Layout
1230 foo.c:3: for each function it appears in.)
1233 \begin_layout Plain Layout
1235 foo.c:3: parse error before `x'
1243 \begin_layout Standard
1244 In order to parse this, the gcc error handler has to look ahead and return
1245 any and all lines that do not match the expected pattern.
1248 \begin_layout Standard
1252 \begin_layout Plain Layout
1254 <<Function Bodies>>=
1257 \begin_layout Plain Layout
1262 \begin_layout Plain Layout
1264 """See if line is a gcc error.
1265 Read ahead to handle all the lines.
1268 \begin_layout Plain Layout
1272 \begin_layout Plain Layout
1274 Returns 1 on success, 0 otherwise.
1275 Outputs on stdout."""
1278 \begin_layout Plain Layout
1283 \begin_layout Plain Layout
1285 <<Handle the gcc error message>>
1288 \begin_layout Plain Layout
1293 \begin_layout Plain Layout
1297 \begin_layout Plain Layout
1307 \begin_layout Standard
1308 The error message starts with a gcc header (as above) without an associated
1312 \begin_layout Standard
1316 \begin_layout Plain Layout
1318 <<Handle the gcc error message>>=
1321 \begin_layout Plain Layout
1323 first_space = string.find(line, ' ')
1326 \begin_layout Plain Layout
1328 if first_space > 1: # The smallest would be "X: "
1331 \begin_layout Plain Layout
1333 if line[first_space - 1] == ':':
1336 \begin_layout Plain Layout
1338 header_to_see = line[:first_space - 1]
1341 \begin_layout Plain Layout
1343 next_line = getline()
1346 \begin_layout Plain Layout
1348 if next_line and next_line[:first_space - 1] == header_to_see:
1351 \begin_layout Plain Layout
1353 num_end = first_space
1356 \begin_layout Plain Layout
1358 while next_line[num_end] in string.digits: num_end = num_end + 1
1361 \begin_layout Plain Layout
1363 if num_end > first_space: # good!
1366 \begin_layout Plain Layout
1368 <<Accumulate gcc error lines and print it>>
1371 \begin_layout Plain Layout
1373 else: # oops! Not a gcc error.
1376 \begin_layout Plain Layout
1381 \begin_layout Plain Layout
1386 \begin_layout Plain Layout
1388 pushline(next_line) # return this line to input stream
1391 \begin_layout Plain Layout
1401 \begin_layout Standard
1402 At the point in the code that we know that we are in the middle of an error
1403 message, we do the following:
1406 \begin_layout Standard
1410 \begin_layout Plain Layout
1412 <<Accumulate gcc error lines and print it>>=
1415 \begin_layout Plain Layout
1417 num_str = next_line[first_space:num_end]
1420 \begin_layout Plain Layout
1422 msgs = [line[first_space:]]
1425 \begin_layout Plain Layout
1427 msgs.append(next_line[num_end + 1:])
1430 \begin_layout Plain Layout
1432 header_to_see = next_line[:num_end]
1435 \begin_layout Plain Layout
1437 next_line = getline()
1440 \begin_layout Plain Layout
1442 while next_line and next_line[:num_end] == header_to_see:
1445 \begin_layout Plain Layout
1447 msgs.append(next_line[num_end + 1:])
1450 \begin_layout Plain Layout
1452 next_line = getline()
1455 \begin_layout Plain Layout
1457 if next_line: pushline(next_line)
1460 \begin_layout Plain Layout
1462 write_error(msgs, "gcc", int(num_str))
1465 \begin_layout Plain Layout
1470 \begin_layout Plain Layout
1480 \begin_layout Subsection
1484 \begin_layout Standard
1485 A xlc error message is easy to identify.
1486 Every error message starts with a quoted string with no spaces, a comma,
1488 \begin_inset Quotes eld
1492 \begin_inset Quotes erd
1495 , a space, and some variable text.
1496 The following routine tests if a given buffer line matches this criteria
1497 (this code would probably be simpler if I used the
1498 \begin_inset Quotes eld
1502 \begin_inset Quotes erd
1505 regexp module, but we don't really need the full regular expression engine
1510 \begin_layout Standard
1514 \begin_layout Plain Layout
1518 \begin_layout Plain Layout
1522 \begin_layout Plain Layout
1524 <<Function Bodies>>=
1527 \begin_layout Plain Layout
1532 \begin_layout Plain Layout
1534 """see if line is an xlc error.
1537 \begin_layout Plain Layout
1541 \begin_layout Plain Layout
1543 Returns 1 on success, 0 otherwise.
1544 Outputs on stdout."""
1547 \begin_layout Plain Layout
1552 \begin_layout Plain Layout
1554 if line[0] == '"': # This is the first character of all xlc errors
1557 \begin_layout Plain Layout
1559 next_quote = string.find(line, '"', 1)
1562 \begin_layout Plain Layout
1564 first_space = string.find(line, ' ')
1567 \begin_layout Plain Layout
1569 if (next_quote != -1) and (first_space > next_quote): # no space inisde
1573 \begin_layout Plain Layout
1575 if line[first_space - 1:first_space + 6] == ", line ":
1578 \begin_layout Plain Layout
1580 num_start = num_end = first_space + 6
1583 \begin_layout Plain Layout
1585 while line[num_end] in string.digits: num_end = num_end + 1
1588 \begin_layout Plain Layout
1590 if num_end > num_start:
1593 \begin_layout Plain Layout
1595 write_error(line, "xlc", int(line[num_start : num_end]))
1598 \begin_layout Plain Layout
1603 \begin_layout Plain Layout
1608 \begin_layout Plain Layout
1613 \begin_layout Plain Layout
1623 \begin_layout Section
1627 \begin_layout Standard
1628 This project can be tangled from \SpecialChar LyX
1630 \begin_inset Quotes eld
1634 \begin_inset Quotes erd
1637 convertor to call a generic script that always extracts a chunk named
1642 Here is an example of such a generic script:
1645 \begin_layout LyX-Code
1649 \begin_layout LyX-Code
1650 notangle -Rbuild-script $1 | env NOWEB_SOURCE=$1 NOWEB_OUTPUT_DIR=$2 sh
1653 \begin_layout Standard
1654 This section defines our build-script, which extracts the code.
1657 \begin_layout Standard
1661 \begin_layout Plain Layout
1666 \begin_layout Plain Layout
1671 \begin_layout Plain Layout
1673 if [ -z "$NOWEB_SOURCE" ]; then NOWEB_SOURCE=listerrors.nw; fi
1676 \begin_layout Plain Layout
1678 if [ -z "$NOWEB_OUTPUT_DIR" ]; then NOWEB_OUTPUT_DIR=.; fi
1681 \begin_layout Plain Layout
1683 notangle -Rlisterrors ${NOWEB_SOURCE} > ${NOWEB_OUTPUT_DIR}/listerrors
1686 \begin_layout Plain Layout
1688 chmod +x ${NOWEB_OUTPUT_DIR}/listerrors
1691 \begin_layout Plain Layout
1701 \begin_layout Section
1705 \begin_layout Standard
1706 This section provides cross-references into the rest of the program.
1709 \begin_layout Subsection
1713 \begin_layout Standard
1717 \begin_layout Plain Layout
1729 \begin_layout Subsection
1733 \begin_layout Standard
1737 \begin_layout Plain Layout