> The biggest thing that could be improved about parsing IMO is for grammars to be reusable.
This is not solely about parsing. There is the semantic analysis phase afterwards, which is just as hard. In your example "sizeof(struct MyStruct) = 96" must be related to the type definition of MyStruct. Then the compilation target must be asked for the sizes of the primitive types inside the struct. Effectively, the IDE could as well execute the compiler (with a parameter not to generate code) and parse the output.
Maybe an IDE wants a complete frontend as a library. Feed in the file, retrieve the AST. Insert callbacks for warnings and errors instead of printing to stderr.
An interesting problem would be to reparse parts of a file. The parser would have to get an AST node and a file position, to "resume" parsing from there. This only works, if there is no context apart from the AST, though.
> Just consider. The simple way of removing a single "i" tag from strings like "<i>help me</i> he said" is never going to be "load XML parser, issue commands, close XML parser". A single loop removing the offending character is going to beat this "algorithm" a thousand times over. Manual parser generation more or less allows you to boil down your parser to this.
However, the parser-way might enable you code to handle tricky XML stuff. E.g. namespaces: "<html:i>help me</html:i>".
> I think parser-generators are unpopular because people would prefer to just write code, rather than compile something else to automatically generated code that is nigh on unreadable.
Also, a manual lexer/parser can introduce context when necessary. E.g. Python has significant whitespace. The lexer (with context knowledge of indentation width) can easily emit indent/dedent tokens, so the grammar is context free. With a tool like ANTLR you have to do wierd stuff to parse Python.
Programming languages older than 10 years or so are usually not context-free. For example C needs context to parse "A*B", because the meaning depends on whether "A" is a type or a variable. Recent programming languages usually try to be LL(1), which is why the keywords "var", "val", and "def" become so popular.
This is not solely about parsing. There is the semantic analysis phase afterwards, which is just as hard. In your example "sizeof(struct MyStruct) = 96" must be related to the type definition of MyStruct. Then the compilation target must be asked for the sizes of the primitive types inside the struct. Effectively, the IDE could as well execute the compiler (with a parameter not to generate code) and parse the output.
Maybe an IDE wants a complete frontend as a library. Feed in the file, retrieve the AST. Insert callbacks for warnings and errors instead of printing to stderr.
An interesting problem would be to reparse parts of a file. The parser would have to get an AST node and a file position, to "resume" parsing from there. This only works, if there is no context apart from the AST, though.