Michael Schubart’s Weblog

1500 Yen = 14.19 US$ = 11.37 Euro

A picture I shot last weekend in Shiodome, Tokyo.

This post on Boing Boing explains a clever way how spammers hack CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart).

Looking at the spam in my inbox, I get the impression that many spammers are not very bright. This one, however, strikes me as very original and sophisticated.

Click above to get the full picture. Found in the Party Car tribe’s photo album.

Many technologies (e.g. XSLT, XML Schema and Ant build files), require the creation of complex XML documents. The complexity of these technologies mandates good comments in order to keep the documents maintainable. These technologies are demanding and it is not trivial to create documents that work as expected. Typically, a developer will create a document, test whether it works, adjust the document, test again and so on. This interactive programming style requires the ability to temporarily comment out passages of the document. So, comments are needed for two purposes:

• Add explanations to code.
• Comment out code.

This post explains that XML comments do not meet these two requirements well. It contrasts XML with Java and shows that Java comments are more programmer-friendly:

XML has only one kind of comment:

• Starting with <!-- and ending with -->.

You cannot easily comment out code that already contains comments:

<!–
  <xsl:template match=”widget”>
    <!– Calculate foo as sum of bar and baz. –>
    <xsl:variable name=”foo” select=”@bar + @baz”/>
  </xsl:template>
–>

This does not do what you might expect. Only the bold part is commented-out. The final --> is mismatched. Also, the comment contains the illegal character sequence --. The above XML snippet is not even well-formed XML.

Java, on the other hand, has two kinds of comments:

• Traditional comments starting with /* and ending with */.
• End-of-line comments starting with // and extending to the end of the line.

You can easily comment out a block of code - even if that code already contains comments. Code that contains only end-of-line comments can be commented out using either traditional comments like this:

/*
public int getFoo() {
    // Calculate foo as sum of bar and baz.
    return bar + baz;
}
*/

Or end-of-line comments can be used like so:

// public int getFoo() {
//     // Calculate foo as sum of bar and baz.
//     return bar + baz;
// }

Typing all these // characters can be tedious, but most modern editors and IDEs will do it automatically. See this page for the respective commands in Eclipse.

If you want to comment out a piece of code that already uses traditional comments, you have to use the second method (end-of-line comments):

// public int getFoo() {
//     /* Initialize transdimensional machinery. */
//     init();
//
//     // Calculate foo as sum of bar and baz.
//     return bar + baz;
// }

Conclusion: XML-based technologies would be easier to work with if XML had end-of-line comments like Java does.

Earlier I talked about my issues with JTidy, the Java version of HTML Tidy. Well, today I found that Fabrizio Giustina has restarted the JTidy project. Some improvements (Maven build file and Maven generated project web page) are already in place. He plans to bring JTidy up to speed with HTML Tidy.

Thank you, Fabrizio!

Seen a few years ago at the Sangubashi station of the Odakyu train line in Tokyo.

I just discovered John Cowan’s TagSoup, an XML parser for Java (SAX interface) that can parse documents even if they are not well-formed. It can also parse HTML documents (which, unlike XHTML documents, are not necessarily well-formed XML).

It should be very easy to implement screen scraping by combining this parser with XSLT:

• Parse the HTML web page with TagSoup, convert it to a well-formed XML document.
• Use XPath Explorer to construct the XPath expressions that select the content you are interested in.
• Use the XPath expressions to write an XSLT stylesheet that converts the web page into a format you like (e.g. RSS).
• Hope that the layout of the web site does not change too much in the future…

I did the above, using HTML Tidy to convert HTML web pages to XHTML, but I ran into the following problems:

• HTML Tidy aborts when encountering non-HTML tags. TagSoup claims that it never throws an exception, no matter what it finds in the document.
• It’s a hassle to run native command line tools from Java programs.
• JTidy, the Java version of HTML Tidy did not work as well as the C version. The last release of JTidy seems to have been in August 2000 or 2001.