I recently needed to load a huge wrl file (zipped it was 63MB). That failed on memory issues.
Then I started to look through the x3d scene importer, and found out what it was doing:

1. read the whole file into a string (!)
2. split the string into lines, do some filtering, and merge that.
3. split the string into lines, do some filtering (place { and [ and ] and } on separate lines) , and merge that.
4. split the string into lines, remove unneeded whitespace , and merge that.
5. split the string into lines
6. start processing the lines.

I found it hard to understand how the parser internally worked (very complicated, and no documentation of the AST that is being generated), but figured out that all the functions are passing line-numbers to each other, telling where in the global array of lines they should be reading.

As far as I could see, it is never really needed to look backward more than 1 line. So I used some generator-functions and iterators to read and cleanup the wrl file line by line, and created an object which behaves like an array to replace the lines array.

However, I don't have a wide range of wrl files to check all the features of the importer, nor do I have x3d files to check if that still works.

This zip file contains an acceptance test, and a small test wrl file (the real files I can't include due to copyright reasons).
It also contains the version of the importing code that was shipped with 2.69, and my version, and tests whether or not the ASTs that are generated by the two, are the same.

To test it, unzip the zip somewhere, paste some wrl files in the folder, open blender, open comparison.py in blender, and add the names of the wrl files in the TESTFILES list.
It will write the output to files, and print a diff in blender console if there are differences.