WPF WebBrowser control – part 3

Showing the Windows 8 touch keyboard

The WPF WebBrowser control does not natively support touchscreens. To make an application easily usable even on touch devices, you need to provide at least one capability: the tablet keyboard must appear whenever the user touches an editable field and moves the focus there.

In part 2 we saw how you can register to DOM events and handle them in .NET code. Ideally, to make the keyboard appear, you will register a handler for every significant field (input, textarea, etc…) and lauch the keyboard process in the body of the callback.

Making the keyboard appear is quite trivial. In Windows 8/8.1, it is as simple as starting the executable at C:\Program Files\Common Files\microsoft shared\ink\TabTip.exe. The following code will do the trick:

    public static void ShowTouchKeyboard()
    {
        using (var keyboard = new Process())
        {
            keyboard.StartInfo = new ProcessStartInfo(@"C:\Program Files\Common Files\microsoft shared\ink\TabTip.exe", "/ManualLaunch");
            keyboard.Start();
        }
    }

Once the field has lost the focus, you may want to hide the keyboard. This is a bit more complicated. First, also for this case, you need to register to the appropriate DOM event on the managed elements (e.g. onblur). Then, you need to rely on the Windows API in order to find the window corresponding to the keyboard process and send it a message to close it. The following code fragment can be added to the WebBrowserAdapter class that we already know from part 2 of this series:

public static readonly int WM_SYSCOMMAND = 274;
public static readonly uint SC_CLOSE = 61536;

[DllImport("user32.dll")]
public static extern IntPtr FindWindow(String sClassName, String sAppName);

[DllImport("user32.dll")]
public static extern IntPtr PostMessage(IntPtr KeyboardWnd, int WM_SYSCOMMAND, uint SC_CLOSE, int p);

public static void ShowTouchKeyboard()
{
    using (var keyboard = new Process())
    {
        keyboard.StartInfo = new ProcessStartInfo(@"C:\Program Files\Common Files\microsoft shared\ink\TabTip.exe", "/ManualLaunch");
        keyboard.Start();
    }
}

public static void CloseTouchKeyboard()
{
    IntPtr keyboardWnd = FindWindow("IPTip_Main_Window", null);
    if (!keyboardWnd.Equals(IntPtr.Zero))
    {
        PostMessage(keyboardWnd, WM_SYSCOMMAND, SC_CLOSE, 0);
    }
}

WPF WebBrowser control – part 2

Registering .NET handlers to DOM elements’ events

With an approach similar to the previous one (see part 1), we can register callbacks that get executed when a DOM event is triggered. Simply put: we can execute .NET code in response, for example, to a click on a HTML input text element. To do this you need to add the MSHTML library to the project references, like I showed in part 1.

As before, let’s assume that we have a simple UserControl that uses a WebBrowser:

<UserControl x:Class="WebBrowserExample.WebBrowserAdapter"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
mc:Ignorable="d"
d:DesignHeight="300" d:DesignWidth="300">
    <Grid>
        <WebBrowser x:Name="WebBrowserControl"></WebBrowser>
    </Grid>
</UserControl>

and we add our UserControl to the main window, just like this:

<Window x:Class="WebBrowserExample.MainWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
Title="MainWindow" Height="350" Width="525"
xmlns:cont="clr-namespace:WebBrowserExample">
    <Grid>
        <cont:WebBrowserAdapter></cont:WebBrowserAdapter>
    </Grid>
</Window>

We can now use types and classes from mshtlm to directly access the DOM of a HTML page. The following snippet reports the complete code of the WebBrowserAdapter class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using mshtml;

namespace WebBrowserExample
{
    /// <summary>
    /// Interaction logic for WebBrowserAdapter.xaml
    /// </summary>
    public partial class WebBrowserAdapter : UserControl
    {
        public WebBrowserAdapter()
        {
            InitializeComponent();
            this.Loaded += WebBrowserAdapter_Loaded;
        }

        void WebBrowserAdapter_Loaded(object sender, RoutedEventArgs e)
        {
            WebBrowserControl.LoadCompleted += WebBrowserControl_LoadCompleted;
            WebBrowserControl.Navigate("http://www.google.com");
        }

        void WebBrowserControl_LoadCompleted(object sender, NavigationEventArgs e)
        {
            HookHTMLElements();
        }

        private void HookHTMLElements()
        {
            var document = WebBrowserControl.Document as HTMLDocument;
            var inputElements = document.getElementsByTagName("input");
            foreach (var item in inputElements)
            {
                DispHTMLInputElement inputElement = item as DispHTMLInputElement;
                if (inputElement.type == "text" 
                    || inputElement.type == "password" 
                    || inputElement.type == "search")
                {
                    HTMLButtonElementEvents_Event htmlButtonEvent = inputElement as HTMLButtonElementEvents_Event;
                    htmlButtonEvent.onclick += FocusCallback;
                }
            }
        }

        public bool FocusCallback()
        {
            Console.WriteLine("Callback method executed!");
            return true;
        }
    }
}

In the constructor, WebBrowserAdapter registers itself as a listener to its Loaded event. This event is fired when the user control is rendered and ready for interaction. In the WebBrowserAdapter_Loaded it begins to prepare the WebBrowser control: first, it subscribes to the LoadCompleted event of the WebBrowser: the WebBrowser fires this event whenever a navigation successfully ends. Then it makes the WebBrowser navigate to the Google home page, calling the Navigate method.

When the WebBrowserControl_LoadCompleted method gets executed, we can be sure that the WebBrowser holds a valid HTML document and we can safely call its Document property and manipulate the object return. This is what we do in the HookHTMLElements method: here we go retrieve all the elements that have tag name equal to “input”, much like we would do in a Javascript function (in fact, if you take a look at the methods the intellisense proposes you, you will see that they match the Javascript APIs), and we attach .NET callbacks to the DOM events. Now, if you click on the input text box of the Google home page, the FocusCallback method is called and a line is written on the standard output (see pictures below).

 

 

WPF WebBrowser control – part 1

Introduction

Recently I spent a lot of time developing a .NET WPF application that embeds a web browser (don’t hold me guilty for that, it’s the client’s requirement). Since the standard WPF WebBrowser has a lot of limitations and problems, I had to find several solutions that are not perfectly standard or ideal for the WPF world. This series of posts presents a summary of those solutions that I find particularly interesting. The majority of the material presented here has been taught to me by other developers or gathered in the Internet, though some solutions are invented by me.

Limitations

I found 2 main limitations of the WPF WebBrowser control:

1. It is not well suited for the MVVM pattern. None of its functionalities is accessible through Dependency Properties, so you have to wrap it somehow if you want to employ it in a MVVM based design. In my case, a decent part of the work has been building an adapter in order to use the WebBrowser in conjunction with the Caliburn.Micro framework.
2. The set of properties and methods exposedbytheWPF control is extremely limited. When you are using theWebBrowser control you are actually using aWPF wrapper of Internet Explorer. However, it hides a lot of functionalities that the actual underlying implementation has. If you want to have the full (?) power of IE at your disposal, you have to work your way around to get to the underlying COM object. The capabilities that are not visible via the .NET interface includes, but are not limited to, the following:
   1. Use a different IE version (rather than the default IE7);
   2. Prevent the browser from showing annoying dialog windows when a Javascript error occurs;
   3. Attacching .NET handlers to Javascript events;
   4. Injecting HTML and Javascript from .NET code;
   5. Calling Javascript function from .NET code;

Using the Internet Explorer 11 engine and not the IE 7 one

One of the easiest tricks to perform is also one of the subtlest. I don’t remember where I learned it but I definitely did not find it myself. To make sure that the actual Internet Explorer component used inside your WPF application is the one from the version 11 (or possibly higher, in the near future), you have to add a couple of keys to the Windows Registry. Open the editor (run regedit) and go to the branch with path: HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_BROWSER_EMULATION (Create the folder if it does not exist). Add a new DWORD value for each executable where the WPF WebBrowser control must use IE11, just like this:

The key of the entry must be equal to the name of the executable file of your interest, while the value encodes the version of IE that the corresponding app will use (the hexadecimal value 0x00002af9 for version 11, in this case). Note that some applications may already be there (I had the Acrobat Reader executable that I left there as an example). Also note that if you are developing the application with Visual Studio you may also want to add the name of the executable launched inside the IDE when debugging (the entry with the .vhost.exe suffix ).

How to inject Javascript into HTML documents

First of all, you are going to need some extra dependencies so better add them immediately. Look at the picture below: you must add the mshtml library. You should find it in the Assemblies -> Extensions section of the Reference Manager dialog window.

Now let’s suppose that you have a class (tipically a Window or a UserControl) that uses a WebBrowser. You should have something that looks like this: (The XAML)

<UserControl x:Class="ExampleWebBrowserWrapper"
 xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
 xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
 xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
 xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 
 mc:Ignorable="d" 
 d:DesignHeight="300" d:DesignWidth="300">

    <Grid>
        <WebBrowser x:Name="webBrowserControl"></WebBrowser>
    </Grid>

</UserControl>     
        

In order to inject a Javascript script into the HTML page, so that it is interpreted and possibly executed in the context of the page, you can implement a method like the following in the ExampleWebBrowserWrapper class (the rest of the class code is ignored for brevity’s sake):

using mshtml;
/* other "using" are omitted */

public partial class ExampleWebBrowserWrapper : UserControl
 {

 public void InjectScript(String scriptText)
 {
 HTMLDocument htmlDocument = (HTMLDocument)webBrowserControl.Document;

 var headElements = htmlDocument.getElementsByTagName("head");
 if (headElements.length == 0)
 {
 throw new IndexOutOfRangeException("No element with tag 'head' has been found in the document");
 }
 var headElement = headElements.item(0);

 IHTMLScriptElement script = (IHTMLScriptElement)htmlDocument.createElement("script");
 script.text = scriptText;
 headElement.AppendChild(script);
 }

}

Then you can call Javascript function (the ones you injected or the ones already contained in the HTML page) from .NET code by calling the InvokeScript method of the WebBrowser class.

        public void InvokeScript(String javascriptFunctionName)
        {
            try
            {
                webBrowserControl.InvokeScript(javascriptFunctionName);
            }
            catch (System.Exception ex)
            {
                /* Handle Exception */;
            }
        }

To the Store and beyond

The Windows Phone 8 GUI of the BacteriaSimulation project has finally made its way to the App Store: http://www.windowsphone.com/en-us/store/app/mobilebact/2502f176-f850-4ba1-8438-0012d58a96ad.

The app stands out with an astonishing zero download count and a troll comment (in italian) by one of my friends. I received several feedbacks, mostly by my colleagues and friends. The most commonly asked question is: “It’ cool but… what is its purpose?”. I’ve learned to answer politely to this one but my first reaction has been something like: are you kidding me? What’s the purpose of chess, what’s the purpose of art? What’s the purpose of anything that’s fun and challenging? If everyone had ever thought like this, we would still live inside caves. “Hey, what’s the purpose of those strange figures you’ve just drawn on the cave’s walls?”. But now the standard answer is more like: “It’s an extensible environment bla bla bla” or, more probably, “it’s fun”.

Another favorite question is “It would be nice if it was possible to (control the bacteria | make an Android version | bring it to 3D | have a better icon | …), why don’t you do that?”. Why don’t you help me doing that? I am “hiring” people to add new features and become rich out of the project. I mean rich. Not joking. Join me.

Let me introduce the mighty bacteria

While I was busy reducing tangle indexes at work and pumping my CV in the free time, I received a pleasant surprise from one of my personal project’s statistics page (the project I am most proud of). In the section called “top referring sites” a link stands out: http://channel9.msdn.com/coding4fun/blog/Playing-with-Bacteria-virtuallywith-the-BacteriaSimulation-project has 173 occurrences — at the time that I am writing. In a project with a bare front page and no documentation, either it is a spam attack or something weird has happened. It turned out that the latter was the case: a guy from Channel9 (and that means MSDN, in my mind), liked my project and wrote a short post about it the Coding4Fun blog. My intention was to do it later but this seems the perfect time to present BacteriaSimulation, a simulation environment for software bacteria.

The project started out as a testbed for practicing Windows Phone 8 applications. It rapidly grew into an extensible simulation environment to be used both as a basis to build mobile games and a system in which running visual simulations of evolving animal populations. At least, this two have become the main objectives of the project.

The project is still in progress, it has some problems (such as the lack of documentation) and some features are not very stable. A Windows Phone 8 app is still waiting to be published to the store. Nonetheless, I look forward to make it better. More news on the project will follow on this blog.