Diary of a VB.NET Compiler Developer

VB 8.0 Spec & Few last words

2005-06-30T22:26:00.000+05:30

It is more than 2 months since I have committed to the BMCS tree and I am less likely to make further commits. (My employer is no Google you see !)

Mono's VB.NET compiler - mbas has seen steady patches in the last 6 months and I wish that it goes production very soon. I would but be proud that it has some of my little commits.

BMCS is up for grabs if someone wants to own it !

The fine art of Articulation

2005-04-11T23:56:00.000+05:30

I had been following the Linux and BitKeeper story.

Newsforge has carried an article on the same.

I find the following argument to be particularly subtle and very well articulated. I quote Linus' own words from the article:

Larry is perfectly fine with somebody writing a free replacement. He's told me so, and I believe him, because I actually do believe that he has a strong moral back-bone.

What Larry is _not_ fine with, is somebody writing a free replacement by just reverse-engineering what _he_ did.

Larry has a very clear moral standpoint: "You can compete with me, but you can't do so by riding on my coat-tails. Solve the problems on your own, and compete _honestly_. Don't compete by looking at my solution."

And that is what the BK license boils down to. It says: "Get off my coat-tails, you free-loader". And I can't really argue against that.

As I juxtapose these arguments with the work that I have been doing with BMCS, I feel shivers running through my "backbone".

Utility of a Open Source VB.NET compiler ?!

2005-04-10T20:56:00.000+05:30

What is the utility of having a VB.NET compiler in the Open Source world ? Is it something that is good to have or is it something that is a must have ? What would be the 'scenario' like when the mono community is ready with a fully working and reliable VB.NET compiler ?

These questions constantly pound me as I work on new patches for the BMCS ...

"Feature Creeps" in BMCS

2005-03-27T20:04:00.000+05:30

This "Good Friday" weekend had been a "Good Hacking" weekend for me. I made good progress on the conversions between the primitive types.

As I was hacking, the following question constantly surfaced at the back of my head - "How will I be managing the features sneakily creeping in from gmcs world to the bmcs world?". There are two components to answering this question - Identifying that a feature is creeping in and tactfully "handling" the same.

To cite an example, "unsigned numeric types" are "feature creeps" from the C# world. The following pattern keeps recurring in the gmcs conversion routines.

static public Expression ImplicitNumericConversion (EmitContext ec, Expression expr,
                               Type target_type, Location loc)
{
    Type expr_type = expr.Type;
    if (expr_type == TypeManager.ushort_type){
        //
        // From ushort to int, uint, long, ulong, float, double
        //
        if (target_type == TypeManager.int32_type)
            return new OpcodeCast (expr, target_type, OpCodes.Conv_I4);
   }
}

The ushort_type above is cached in the TypeManager as follows:

public class Typemanager
{
    // cached unsigned short type
    static public Type ushort_type = typeof (System.UInt16);
}

One naive way of handling this is to remove all references to the unsigned types. I tried doing this snip, snip, snip ... and it's no fun but a plain drudgery.

Then I sat back and said to myself, sure enough, I can do better than this and ended up doing this -

namespace Mono.CSharp {
    public class NotDefinedAsPrimitiveType {
    }
}

public class TypeManager
{
    public static Type ushort_type = typeof (NotDefinedAsPrimitiveType);
}

Barely three lines of change and most of the feature creep is gone !

Btw, I should admit that it took more time writing these few lines of code than doing all those snips ...

Coming back to the question that I raised previously, the obvious answer is that I should start educating myself about the language semantics in the C# world. This will help me make educated and intelligent choices for handling of feature creeps.

A Reference VB Parser

2005-03-24T18:21:00.000+05:30

Paul Vick has earlier released a VB Parser under the Shared Source Licence. The parser is written in VB.NET and is apparently of beta 2 quality complying with 7.1 version of the spec.

The above parser could help Mono's VB.NET compilers (mbas and bmcs) in two ways:

A reference implementation against which ambiguities in the language specification could be clarified
As a test suite in itself

Challenges in writing a compatible VB.NET compiler

2005-03-12T21:15:00.000+05:30

Thanks to this user comment to my previous enty , I am realising that I need to be more aware of the happenings on in the VB.NET world.

Anyways, to answer this reader's comment, I should admit that my immediate focus is to get a compiler that complies to the VB.NET 7.1 Language Specification . When I get to grab a copy of the VB.NET 8.0 Language Specification, I would start getting in the related changes in to BMCS. Can anyone of you make this document available to me if at all Microsoft has made this available as part of their .NET 2.0 bundle ?

On a related note, let me register here that there are two "must haves" for making available a fully functional and compatible VB.NET compiler.

The Languages Specification
A functional and compatible "VB.NET Runtime"

The item (1) above is obvious in itself and it is indisputable that it is a "must have".

The item (2) above is less obvious and requires some elaboration and explanation. "VB.NET Runtime" broadly refers to a set of service routines that is implicitly made use of in any VB.NET executable. These service routines provide type conversion and late binding related functionality and is tucked away in the Microsoft.VisualBasic.CompilerServices namespace of the Microsoft.VisualBasic.dll. Any developer who had casually peeked in to a VB.NET execuatble would have noticed that Microsoft's VB.NET compiler - vbc sneaks in calls to the above namespace to provide the "VB.NET experience" to the programmer. So, a certain amount of base compiler functionality is made available as part of the Microsoft.VisualBasic.dll

The "VB.NET runtime" is a bit troublesome in that the Compiler Services APIs are not at all documented as part of Microsoft's SDK. As a result considerable reverse engineering needs to be done to get their functionality right.

I am particularly happy to note that both the above two "must haves" are taken care of vis a vis Mono's .NET 1.1 environment.

I am thankful to folks at Microsoft for having made the VB.NET 7.1 spec freely and widely accessible.

As for "VB.NET runtime", much of the groundwork had been done by Chris J Breisch, Rafael Teixeira, Martin Adoue, Dennis Hayes and Duncan Mak. Special thanks are due to folks at Mainsoft who made available a huge chunk of Java code that implements the whole of Microsoft.VisualBasic.dll.

It remains to be seen how the Mono community goes ahead and meets the above two challenges wrt VB.NET 8.0 language environment.

Btw, I thank Ankit Jain , for submitting the much required perl fixups to Mono's VB.NET runtime. The inexpressibility of "Default Parameter Values" in C#-based Microsoft.VisualBasic.dll has at last been addressed. I am waiting to see the corcompare results for this critical library once the patches have been successfully merged.

Type Conversions in BMCS

2005-03-05T23:10:00.000+05:30

BMCS now has the basic infrastructure in place to do VB.NET like Type Conversions - both Implicit and Explicit.

The changes turned out to be much simpler than I had earlier thought.

The main difficulty was the nomenclature used by the C# and VB.NET specifications. It took sometime for me to realize the following two facts:

VB.NET 's notion of "Widening and Narrowing Conversions" is equivalent to C# 's notion of "Implicit and Explicit Conversions".
VB.NET has no notion of "User Defined Conversions".

Once the above essential facts were in place, a quick renaming of gmcs's conversion methods and a little bit of refactoring put everything in perspective. (Whoever asked "What is in a name?" should definitely be a fool!)

One biggest challenge introduced by the above renaming and considerable deletions (VB.NET has no unsigned data type other than "Byte", you see) is the increased lexical distance between the gmcs and bmcs trees.

I was immediately tempted to assess how much of a overhead that it introduces in merging of the two trees. After my second merge, I would say that it makes my life more difficult. Eventhough merging in itself is more of a pain, verifying that it indeed is just fine is actually much easier - just do a "diff of diffs" and verify that the only diffs are in the "positions of the diff".

Once I am done with giving final touches to the conversion changes, I think I might resort to "re renaming" the methods. (Whoever asked "What's in a name?" definitely has his feet firmly on the ground!)

My next goal is to VBify expression semantics and follow that up with a "rigorous" article on how C# and VB.NET differ with respect to Type Conversions.

Going online

2005-02-27T23:38:00.000+05:30

After a long wait, I have gotten what I have always missed. An internet connection at home.

I now have more access to one and all.

I am more accessible now.

Lessons in keeping GMCS and BMCS in sync !

2005-02-13T12:44:00.000+05:30

Thanks to this nice article on Building Mono on Windows, I had my build environment set up within a Cygwin environment on my Windows XP box and I was able to get my first set of "gmcs merge patches" in to bmcs.

(Paco, Thanks dude ! You are doing an excellent job in making my "Mono hacking" more productive and less hassle free. It gives me ready access to the benchmark implementation - Microsoft's vbc - while fixing up bmcs.

From this first ever merge, I could see the following challenges that lie ahead in keeping the gmcs and bmcs trees in sync:

The merging of trees should happen as frequently as possible. Checkpointing every two weeks is a must till such time as bmcs begins bloating. Any longer intervals of merges between the two trees would become unmanageable.
The feature additions to bmcs should itself be made in batches as opposed to increments so that one could keeping the lexical distance between the two trees to the minimum.
The VB.NET specific changes should be properly documented either through visible comments or conditional compiler constants so that one doesn't end up losing those changes while performing a merge.

The above challenges need to be delicately managed, as otherwise bmcs might end up being a significantly different beast compared to gmcs. This would seriously hamper bmcs from borrowing liberally from gmcs.

Why BMCS and Why not MBAS ?

2005-02-04T20:39:00.000+05:30

Earlier I had blogged about my intention to focus my spare time on contributing to bmcs. Therein I had declared my intention to make bmcs as "The future defacto Open Source VB.NET compiler".

An interested VB.NET user has raised the following questions - "This will delay a lot the VB.NET compiler development. All mbas compiler work will be discarded?"

The concerns of this VB.NET user is very valid but is totally unfounded. This is because folks from Novell - Manjula, Satya Sudha with additional help from Sudarshan and Ritvik Mayank - are continuing their work on mbas and making it better day by day. So mbas is not only thriving, it is all the more likely to see the light of the day much sooner than bmcs.

Some people might be tempted to think of bmcs as being positioned in direct competition with mbas. Let me clarify that it's actually the other way round. bmcs will coexist with mbas and try to leverage as much work done on mbas as possible. The birth of bmcs needs to be viewed as one creating a better, more futuristic and robust choice of for the Open Source VB.NET developers.

Let me explain why bmcs is "better", "robust", "scalable" and "futuristic".

bmcs is grown from a more mature C# compiler code base that has already seen the light of the day. If one carefully happens to look at the history of mbas, one would realize that it was also cultured out from the mcs code base - but one that's more than 2 years old. So the code emission modules in bmcs is that much more thoroughly tested and feature rich as compared to the one available in mbas.

bmcs is "futuristic". This is because bmcs is grown out of gmcs - Mono's next generation compiler supporting Generics and Partial Classes in C#. Hence bmcs can start supporting these features almost out of the box in no time.

To appreciate why bmcs is more "scalable", one need to carefully consider how mbas is going to mature. As someone, who had spent significant time working on mbas, I see development on mbas proceeding on the following two levels - neither of which is non trivial.

Purging of C# language semantics with VB.NET language semantics
Making the code generation and emission modules more robust and scalable

The task (1) above is something that needs to be done from scratch for both mbas and bmcs.

The only sensible alternative to task (2) would be to borrow liberally from the existing mcs or gmcs code base. That's easier said than done. This is because the class orgnization in mbas is very old and the class structure of mcs has evolved in to a totaly different beast since the fork. The differentiation has been rendered more pronounced by the aggressiveness with which mcs had grown in the intervening two years.

So bringing in new features like generics and partial classes means a significant refactoring and the attendant testing on mbas code base. Any sensible developer could see the enormity of this task.

It's in relation to task (2) that bmcs is going significantly outwit and outlive mbas . It can not only leverage the work that is being done on mbas but can also leverage the work done on the generics C# compiler. A periodic merging of patches is all that's required to make the later task possible.

Personally I am very upbeat about bmcs. The mood is as well shared by Rafael Texeira - the brain behind the Mono's VB.NET compiler

Hello VB.NET World !

2005-01-26T11:32:00.000+05:30

I am an ardent Open Source enthusiast and an avid Mono hacker.

I had earlier worked on Mono's VB.NET compiler - mbas.

Recently, I have embarked on an initiative to build a VB.NET compiler "entirely from scratch". The initial work has been imported as a new tree - mcs/bmcs in to Mono's repository. Here is the posting from Miguel announcing this initiative.

My goal is to nurture bmcs and grow it as a de facto Open Source compiler for the VB.NET language.

You would get to hear from me on a regular basis through this column.