Ticklish Techs

Lately I was looking for a way to translate geo coordinates (latitude and longitude form my gps device) to some sort of text or names. As you know I’m trying to geotag my photos using a gps logger and PhotoTagStudio. This data can be used to visualize the place a photo was taken on a map (e.g. Google Maps or flickr!). The next step can be to use this data to describe the photo as well.

I think one should be able to translate a geo coordinate into at least the name of the county, the state and the city; maybe to the district or a street name, too. To achieve this I found the geonames.org webpage. This page collects exactly the data I need and allows searching via the website and a webservice.

The webservice itself consists of approximate 30 REST methods one can call by querying an URL. Most of these calls return a XML document (or optional a JSON document). Unfortunately  there is no SOAP / WSDL version of this webservice. For a .net and C# developer a SOAP webservice is very easy to consume but for such a REST / XML webservice you have to create the URLs yourself and parse the resulting XML document.

I started to create a library to query the webservice. This library provides a few static methods mapped to the most useable webservice-methods and returns good shaped .Net objects. These objects carry all the data coming from the webservice. At the moment this library is in an early beta status but you can use it in your own projects. It can be found on Codeplex: GeonamesDotOrgDotNet, there you will find a small example and a demo application. When using this library you should take a look at the documentation of the geonames.org webservices since the interfaces are nearly the same.

I hope this is useful for someone. I will extend the library in the next weeks. Feature requests are welcome, so is help on the coding side.

Currently the following webservice calls are supported:

• findNearby
• findNearbyPlaceName
• findNearByWeather
• findNearbyWikipedia
• get
• hierarchy
• children
• gtopo30
• srtm3

At the end a few random screenshots and a code snippet. For more information please take a look at the Codeplex project page.

// finding some place in Berlin, Germany
// (using prefix m for decimal literals)
Geoname Place =
GeoNamesOrgWebservice.FindNearbyPlaceName(52.51m, 13.4m);

// getting all parents
IEnumerable ParentPlaces = Place.Hierarchy();

// display the partents
// (will print out something like:
//  “Globe > Europe > Germany > Land Berlin > Berlin >”)
foreach (Geoname x in ParentPlaces)
Console.Write(x.Name + ” > “);

The Classes of the Library:

A screenshot of the demo application to test the library:

I created two DirectiveProcessor for Visual Studio to use in the T4 system. I added the classes to JaDAL hoping someone can use them in her project or use the code as an example when creating new DirectiveProcessors.

Let’s start with a short introduction to DirectiveProcessors. DirectiveProcessors are used in .tt-files to provide data to the template. The DirectiveProcessor can take parameters from the declaration in the .tt-file and adds some code to the compiled template. This code (often properties) can be used from within the template. For example the DSL Tools are generating one DirectiveProcessor for each DSL model. In a template form the DSL Tools you will see a line like the following:

<#@ Language1 processor=“Language1DirectiveProcessor”
requires=“fileName=’Sample.mydsl1′” #>

This line uses the Language1DirectiveProcessor and provides it with one parameter (fileName). The DirectiveProcessor adds code to the template to open the given file and creates (in this case) a ExampleModel-property to use in the template code.

I created two DirectiveProcessors to use the data of simple Xml-files and Visual Studio Project files in the templates. The code of the two DirectiveProcessors is very straight forward and maybe one could advance it (e.g. make it compatible with templates written in Visual Basic).

Using the XmlFileDirectiveProcessor

Add a line like the following to your .tt-file:

<#@ XmlFile processor=“XmlFileDirectiveProcessor” FileName=“example.xml” #>

Inside this template you can access the (full qualified) filename via the this.XmlFileName-Property and the Content of this file (as a XDocument) via the this.XmlFile-Property.

Using the VsProjectFileDirectiveProcessor

Add this line to your template:

<#@ ProjectFile processor=“VsProjectFileDirectiveProcessor” FileName=“x.csproj” #>

A property named this.ProjectFile will be added to the template. This property provides you with an instance of the VsProjectFile-class containing the project file contents. This class contains only very little functionality (feel free to add some more and submit it back to me!). Just take a look at the source code of this class. The method GetAllFiles() returns a string array of all files found in the project (supporting C++ and C# project files – .vcproj and .csproj).

Setup

An entry in the registry is needed to allow the T4 system to find custom DirectiveProcessors. Just add the following entries to your registry and don’t forget to point to the right location of the JaDAL.dll.

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0Exp\
Configuration\TextTemplating\DirectiveProcessors\VsProjectFileDirectiveProcessor]
“Class”=”BenjaminSchroeter.Dsl.DirectiveProcessors.VsProjectFileDirectiveProcessor”
“CodeBase”=”D:\\JaDAL\\bin\\Debug\\JaDAL.dll”

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0Exp\
Configuration\TextTemplating\DirectiveProcessors\XmlFileDirectiveProcessor]
“Class”=”BenjaminSchroeter.Dsl.DirectiveProcessors.XmlFileDirectiveProcessor”
“CodeBase”=”D:\\JaDAL\\bin\\Debug\\JaDAL.dll”

These registry keys are for the Experimental Hive of Visual Studio. To register the DirectiveProcessors for the normal Visual Studio instance the registry key starts with HKEY_LOCAL_MACHINE\Software\Microsoft\VisualStudio\9.0
\TextTemplating\DirectiveProcessors\

There was not that much going on here lately. Wolfram and I are very busy these days, but we didn’t forgot about the blog. There are many ideas for articles but to few spare time to write them. Fortunately this will become a little bit better in a few weeks.

For the meantime I build a nice tagcloud to remember you what’s usually going on her:

Produced with Wordle. Images of Wordles are licensed .

Stay tuned!

Every DSL you create with DSL Tools has a model explorer. This model explorer is a tool window in Visual Studio displaying the elements of your model in a hierarchical way. This is often a nice feature but sometimes a hierarchical view of your data is not appropriate. So I came to the question: How to remove the explorer from the generated code?

I could not find any option in the DSL design to remove the model explorer, but if you look at some of the .tt files you will find somewhere a query for this.Dsl.Explorer != null. For example in the package.tt file that generates the package.cs which is responsible for registering the tool window for the model explorer:

<#
    if(this.Dsl.Explorer != null)
    {
#>
    [VSShell::ProvideToolWindow(         typeof(<#= dslName #>ExplorerToolWindow),          MultiInstances = false,          Style = VSShell::VsDockStyle.Tabbed,          Orientation = VSShell::ToolWindowOrientation.Right,          Window = "{3AE79031-E1BC-11D0-8F78-00A0C9110057}")]
    [VSShell::ProvideToolWindowVisibility(         typeof(<#= dslName #>ExplorerToolWindow),          Constants.<#= dslName #>EditorFactoryId)]
<#
    }
#>

Even if I did not find any option to set the Dsl.Explorer property of the model to null (you did not see the Explorer property anywhere in the DSL diagram) it seems that the developers of the DSL Tools had this use case in mind.

To remove the Dsl.Explorer from your DSL model open the .dsl file with a text editor and go to the end. There you will find some XML tags like the following:

<Explorer ExplorerGuid="6c276297-6acd-4e9a-8740-b61ba834004b"  Title="HardwareDescription Explorer">
    <ExplorerBehaviorMoniker      Name="HardwareDescription/HardwareDescriptionExplorer" />
</Explorer>

Just delete these three lines and generate your code once again. Maybe you should reset your Experimental Hive, too.

I tested a DSL with a removed Model Explorer without any problems. It seems the developers of the DSL Tools did a very good job on the code generation templates . The generated ModelExplorer.cs file contains only a single line:

// This source file is empty because    this DSL does not define a model explorer.

That should be a good proof that a DSL can run without the model explorer even if it might be not supported by the DSL Tools.

We all know and love the foreach keyword in C# and use it for loops every day:

List<int> list = new List<int>();
/* fill the list */
foreach (int i in list)
    Console.WriteLine(i);

But the generic List<T> class contains a useful method for small loops, too. The method is called ForEach() and has only one parameter: an Action<T>.

delegates: Action<>, Func<> and Predicate<>

The Framework 3.5 defines a few generic delegate types for actions and functions. In this case a function is something that returns a value (think back to your math class) and actions are something that do not return a value but do something (maybe think back to physics class). All these delegates are defined with zero, one, two, three and four parameters:

public delegate voidAction();
public delegate voidAction<T>(T obj);
public delegate voidAction<T1, T2>(T1 arg1, T2 arg2);
public delegate voidAction<T1, T2, T3>(T1 arg1, T2 arg2, T3 arg3);
public delegate voidAction<T1, T2, T3, T4>(T1 arg1, T2 arg2, T3 arg3, T4 arg4);

public delegateTResult Func<TResult>();
public delegateTResult Func<T, TResult>(T arg);
public delegateTResult Func<T1, T2, TResult>(T1 arg1, T2 arg2);
public delegateTResult Func<T1, T2, T3, TResult>(T1 arg1, T2 arg2, T3 arg3);
public delegateTResult Func<T1, T2, T3, T4, TResult>(T1 arg1, T2 arg2, T3 arg3, T4 arg4);

And there is one more delegate. The Predicate<T> (philosophy class, you know). A predicate comes to a logical decision (true or false; bool return value) based on a single object:
public delegate bool Predicate<T>(T obj);

All these delegates are generic so you can use them in a type save way with the type parameters you need.

Back to the ForEach() method

As I explained above, the ForEach() method takes an Action<T> parameter. In my example this is an Action<int> delegate:

list.ForEach( delegate(int i) {Console.WriteLine(i);} );

This anonymous delegate declaration is not that nice, but we can use Lamda types here:

list.ForEach(i => Console.WriteLine(i));

But let’s take a look at the method we’re calling: void Console.WriteLine(int). That is exactly the definition of the needed Action here. So we can write the code line even shorter:

list.ForEach( Console.WriteLine );

I like this code. It is very short, elegant and readable. It says: “For each [entry of] the list: write it out”. Great.

There is a known issue in the current version of the DSL Tools while dealing with custom TypeConverters or custom TypeDescriptors:

1.10 TypeConverters and TypeDescriptors are not picked up during the build process or during toolbox initialization.
When adding a custom TypeConverter or TypeDescriptor and then building the DSL, the TypeConvertor or TypeDescriptor is not picked up. The workaround is to rebuild the solution with a clean build.

[see Known Issues for Microsoft Visual Studio 2008 SDK 1.0]

Some time ago I posted a workaround for this problem with the TypeConverter, but today a realized that this workaround does not work with the same issue for TypeDescriptors.

Imagine some class with the following attribute:

[TypeDescriptionProvider(typeof(MyClassTypeDescriptionProvider))]
public partial class MyClass : ModelElement
{}

The corresponding TypeDescriptor (provided by the MyClassTypeDescriptionProvider class) is loaded only the first time you build or rebuild your solution. Every time you start Visual Studio after that, this attribute seems to be ignored.

Fortunately there is another way to glue TypeDescriptors to your classes using a static method of the TypeDescriptor class at runtime:

TypeDescriptor.AddProvider(
   new MyClassTypeDescriptionProvider(), typeof(MyClass));

I think a good place for this code is the static constructor of the MyClass type:

partial class MyClass
{
    static MyClass()
    {
        TypeDescriptor.AddProvider(
            new MyClassTypeDescriptionProvider(),
            typeof(MyClass));
    }
}

Another interesting point is: Not only custom TypeDescriptors you write yourself are affected by this problem, also the TypeDescriptors that are generated by the DSL-Tools from your DslDefintion have to struggle with it:

In the DSL Explorer you can define custom TypeDescriptors for each Domain Class and each Shape. Even these TypeDescriptors will not be loaded after the first run. In other words: the definition of TypeDescriptors in the DSL Explorer is pretty useless as long as you do not add the three lines of code to each class. Of course this is something one can automate! I stole some code from PropertiesGrid.tt and added a few lines to create the RegisterTypeDescriptor.tt. Just add this file to your Dsl Project in the GeneratedCode folder and all TypeDescriptors defined in the DSL Explorer will be loaded every time you start your project.

I love CodeVeil. From my point of view CodeVeil is simply the best tool to protect .NET assemblies from being reflectored, decompiled or whatever. It is very powerful, offers a wide range of protection aspects and still comes with a handy gui. So far I haven’t seen a method to crack veiled assemblies and that’s why I really rely on it for my commercial products.

You don’t find bugs in CodeVeil when you try to find them, but half a year ago we found this one by chance. CodeVeil veils well, but the resulting code throws a mighty exception.

You don’t need spectacular code to lure the bug out of it’s hole – here is the prelude:

private static void Main(string[] args)
{
    // Get list and fill it with values
    List<Capsule<int>> l = new List<Capsule<int>>();
    l.Add(new Capsule<int>(23));
    l.Add(new Capsule<int>(42));

    // Have it sorted
    List<Capsule<int>> l2 = GetSortedReturnList(l);

    // Show result
    foreach (Capsule<int> ci in l2)
        Console.WriteLine(ci.Item);

    Console.ReadLine();
}

A simple list of two ints that are encapsulated in a generic type. The generic type just makes sure that two items can be compared:

internal class Capsule<T> : IComparable<Capsule<T>>
{
    public T Item;

    public Capsule(T i)
    {
        Item = i;
    }

    public int CompareTo(Capsule<T> other)
    {
        return Item.ToString().CompareTo(other.Item.ToString());
    }
}

And this code causes the exception when being executed by the veiled assembly:

private static List<Capsule<T>> 

    GetSortedReturnList<T>(List<Capsule<T>> list)
{
    int sortOrder = 1;
    list.Sort(
        delegate(Capsule<T> a, Capsule<T> b) 

          { return sortOrder * a.CompareTo(b); });

    return (list);
}

The sore point is the sortOrder variable being multiplied with the result of a.CompareTo(b) used in an anonymous method. Okay, you wouldn’t expect this code from a beginner, but it’s not rocket-science either. So we were very astonished and it took us quite some time to track this down and to believe our eyes. Maybe the problem with CodeVeil is related to the internal representation of anonymous method. When you use a constant instead of a variable

const int sortOrder = 1;

the problem disappears. I guess the compiler eliminates the constant and CodeVeil will not see it. But it remains, when you use a Linq-notation here:

list.Sort((a, b) => a.CompareTo(b)*sortOrder);

As said before we found this bug half a year ago. We contacted Xheo and received a very quick and friendly reply, but no solution. Seasons went by and several new versions of CodeVeil were released; but this bug is still present. At least in the stand-alone version.

But there is hope: The new beta preview of the DeployLX suite contains an obviously new version of CodeVeil and the bug is finally gone. And – by the way – this new version seems to offer a nice package of new feature and improved power! So, please, Mr. Xheo, release a new stand-alone-version of CodeVeil based on the new beta preview!

As you know I was working with the DSL Tools in the past months. For some weeks I am writing the code generation for my project and I am struggling with new problems.

For some reason I want to know the filename of the model used in the tt-file. The default implementation provides you only with a reference to the model but with no chance to get its filename.

In your tt-file you will find something similar to

<#@ Language15 processor="Language15DirectiveProcessor"
     requires="fileName='Sample.mydsl1'" #>

This will use the Language15DirectiveProcessor to load the given file and provide your template with a global ExampleModel variable (of cause the name depends on your DSL and you can change it with the provides attribute in the tt-file).

To change the Language15DirectiveProcessor you can create a partial class in your DSL and add some code to it:

partial class Language15DirectiveProcessor
{
    protected override void GenerateTransformCode
     (string directiveName,
      StringBuilder codeBuffer,
      System.CodeDom.Compiler.CodeDomProvider languageProvider,
      IDictionary<string, string> requiresArguments,
      IDictionary<string, string> providesArguments)
      {
        base.GenerateTransformCode(directiveName,
                    codeBuffer, languageProvider,
                    requiresArguments, providesArguments);           

        codeBuffer.AppendFormat(
           "public string {1} = @\"{0}\";",
             requiresArguments["FileName"],
             providesArguments["FileName"]);

       codeBuffer.AppendLine();
    }

    protected override void InitializeProvidesDictionary
    (string directiveName,
      IDictionary<string, string> providesDictionary)
    {
        base.InitializeProvidesDictionary(directiveName,
                            providesDictionary);

        providesDictionary.Add("FileName","FileName");
    }
}

This adds a new global variable FileName to the tt and initializes it with the model file name.

Attention: You see the variable in this example is generated by writing a line of C# code to the codeBuffer. This may be a bad idea if you want to use this DirectiveProcessor for templates where the template language is set to VB. The better way is to create the code using the Emit and CodeDom API, but I was to lazy to do it that way.

If anybody ports this code to work with VB templates by using the CodeDom or plain VB code, please post a comment here.

Further reading: The process of creating your own DirectiveProcessor and everything you need to understand the code above can be found in the msdn: Creating Custom Text Template Directive Processors.

One of my DSLs uses many black connectors laying all over the screen. Every time I try to follow one connector I select it to change the border and to see a difference between all connectors. A colleague saw this design and asks me to implement a mouse over feedback for these connectors since all shapes have already such a visual feedback. The color of shapes changes a little bit if the mouse is over the shape. I had no idea how this should work for connectors…

First I take a look on the properties of the connector in the DSL designer and while searching I observed something: The DSL designer has this behavior on all connectors! I don’t have to write code for the MouseOver-event, but I have to figure out how to activate this behavior on my DSL editor.

For my surprise, the connectors on a newly created example language have a mouse over feedback, too. After playing around with some properties I was faced to a interesting fact: The mouse over feedback works pretty good for all connectors but black ones. I had no idea what’s going on.

Long story short: With Reflector I found one interesting method in the ShapeElement class: ModifyLuminosity(). The documentation is very clear:

"Calculates the luminosity of the highlight for the shape."

"The formula that creates luminosity. By default, the formula is: if luminosity is >= 160, then luminosity = luminosity * 0.9; otherwise, luminosity = luminosity + 40."

The luminosity of a black connector is 0 so the new luminosity will be 40. But for the color black you hardly see a difference between 0 and 40. But with this knowledge you can override the method in the connector class like this:

protected override int ModifyLuminosity
              (int currentLuminosity, DiagramClientView view)
{
  if(!view.HighlightedShapes.Contains(new DiagramItem(this)))
     return currentLuminosity;

 return 130;
}

The value 130 on a black connector works pretty good for me. Just try a few values und find a value you like.

It took me a lot of time to find a neat explanation of asymmetric encryption. Many sides say “a message is encrypted using a public key and it can only be decrypted with a corresponding private key.” Okay, fine, but how does it work in detail? Finally I found the RSA-entry in wikipedia. It’s almost what I was looking, but there are still lots of links to mathematical definitions and calculations you can’t do with your pocket calculator. So I decided to write a little compilation of the presented algorithm and to apply it using numbers you can handle. Also small number help to clarify things – but keep in mind that all the numbers are much much much greater in serious encryption.

How do we get our private-and-public-key-pair?

1. Choose two distinct (large) random prime numbers p and q
   p = 7,   q = 3.
   
2. Compute n = p*q. This n is used as the modulus for both the public and private keys
   n = 7*3 = 21.
3. Compute the totient: phi(n) = (p-1)(q-1)
   phi(21) = (7-1) * (3-1) = 12.
4. Choose an integer e such that 1<e<phi(n) and that e and phi(n) share no factors other than 1 (i.e. e and phi(n) are coprime.). e is released as the public key exponent.
   In order to chose an e I will factorize my phi(12) first: 12 = 2*2*3.
   I chose e=5. e does not necessarily have to be prime but it makes it easier to avoid sharing a factor.
5. Compute an integer d to satisfy the congruence relation d*e mod phi(n) = 1; d is kept as the private key exponent.
   “congruence relation” sounds more complicated than it actually is: Take two different numbers and apply the same modulo-operation to them (like mod 3). If the result is the same for both you may call your integers congruent modulo 5.  Like 11 and 16 are congruent modulo 5, since 11 mod 5 = 1 and 16 mod 5 = 1.
   So we are looking for a integer d that fits into
   d * 5  mod 12 = 1.
   I needed some tries here and wrote some lines to find the lowest d = 5. With one d found you can find all but just adding 12 . I didn’t want to take 5 as the private key exponent, since having the same exponent for encryption and decryption is… well… stupid. So I’ll choose 17.
   17 * 5 mod 12 = 1.
   85 mod 12 = 1.  Correct!

Do we have our private and our public key now? Yes, we have. Note that a key is not one single number but a pair of two numbers. Both are needed in the processes of encryption und decryption. One is the exponent and the other the modulus. Just a second and you’ll see why.

public key: e =5 (exponent) / n = 21 (modulus).

private key: d = 17 (exponent) / n = 21 (modulus). 

Let’s encrypt something

We have given our public key to anyone we know. And we kept our private key hidden somewhere under the bed. Now a beautiful and clever girl named Alice wants to send me a message. Fortunately the letters of her message can be represented as a stream of bits and we interpret this stream as a number. So her message is 10. It is important that her message is lower than our modulus, we’ll later see why. Let’s call her original message m and the encrypted message c.

m=10

The encryption it a simple formula: c = m^e mod n.

c = 10^5 mod 21

c = 100000 mod 21

c = 19.

So Alice hands me a little note saying “19″. 19? What the hack is that supposed to mean?

Now the magic happens

I rush back home to find my private key and apply it to “19″

The formula for the decryption look very similar: m = c^d mod n.

m = 19^17 mod 21.

m = 5480386857784802185939 mod 21. (Try it using calc.exe)

m = 10!

Wow! Alice says “10″ to us. Isn’t 10 the international code for “I would like to date you?” I think so.

Signing

To encrypt with the public key means you can decrypt only with the private key. The converse is also true – to encrypt with the private key means you can decrypt only with the public key. Try it!

How can utilize this? We can use it to guarantee that a message is from a specific sender.

I have been dating Alice for some time now and we are used to leave messages to each other at a hidden place. Unfortunately another girl – Eve – is very jealous and has spied our secret mailbox. One week ago she found a message from Alice to me. She couldn’t read it, but she threw it away and replaced it by a mean offense against me. She encrypted it using my public key (which is public ) and I thought it was written by Alice. The message started a little fight but luckily we found about Eve’s intention and started signing our messages. Now we transfer messages like this:

1. Alice writes me a message m.
2. She encrypts m using my public key, the result is c.
3. Now she encrypts c using her own private key, the result is cs (c signed)
4. She leaves me the message
5. I decrypt cs using her public key, getting c. (Eve can do this as well – but that’s it. She can’t read c and she can’t leave me a fake cs because she doesn’t have Alice’s private key.
6. I decrypt c using my private key and get back m – bingo!

Padding

Breaking this code (meaning “decrypt without having the private key”) is always a lot of work and hard trying. But is becomes much easier, when you know that m is short. That’s because you only have check that little fraction of configurations where m = c^d mod n leads to small ms. To avoid this small messages are artificially made longer until they are close to n. That process is called padding and even padding can be a tricky task.

Last question

We know what happens to short messages. But what happens to long messages? It is obvious that messages – seen as a number – can’t be greater than n-1, since the mod n – part of the decryption formula will never return values >n-1. I guess the message is split up in parts, but couldn’t find a satisfying answer. If I do, I’ll let you know.

Take care, 
Bob