Real time logs with Chrome dev tools and signalr part 2


This is the second post in a series talking about creating real time logging using chrome dev tools and real time communication libraries such as signalr. The first post focused on the server-side portion of the setup. This post will focus on creating the chrome devtools plugin which will display the logging information from the server.

About chrome plugins
If you know html/javscript/css, creating a chrome extension is actually really easy. The only gripe I have is there seems to be no way to inspect dev tools extension panels.But, you can get around that by sending errors from window.onerror and try/catch to the inspected window or background page console. Another thing to keep in mind is certain features will not work if you don’t have the appropriate permissions in the plugin configuration. I strongly suggest reading the chrome developer documentation for a better understanding of how devtools plugins work.

Creating the Plugin
I will start off with a layout of the plugin files in the file system and explain each file in a logical order.
Plugin layout

Plugin Manifest
This file tells chrome about the plugin and the various files it needs to work correctly.

{
  "manifest_version": 2,
  "name": "Real Time Logger",
  "description": "This extension allows applications to send logs to client without embedded scripts",
  "version": "1.0",
  "background":{
  	"persistent": true,
    "scripts": ["lib/jquery-2.0.3.js","lib/jquery.signalR-2.0.1.js","background.js"]
  },
  "permissions": [
    "tabs", "http://*/*", "https://*/*"
  ],
  "devtools_page": "devtools.html"
}

The “background” directive will instruct chrome to load an html page and include the three js files as scripts. Alternatively, you can create your own background.html and include the scripts yourself. The permissions control access to otherwise limited capabilities of the chrome extensions api. The devtools_page is where the plugin will create the panel used by the plugin to display the log information.

Background.js
This is the workhorse of the plugin. It will maintain all the connections to the server, receive the log messages and pass them out to the respective panels to be displayed.

var connectionlib = {
	signalr: function(){
		var connection;
		return {
			init: function(settings, handler){
					 var url = settings['baseurl'] + settings['url'];
					 connection = $.hubConnection(url, { useDefaultPath: false });
					 var proxy = connection.createHubProxy(settings['hub']);
					 proxy.on('onSql', function(sql) {
						handler(sql);
					 });
	
					 connection.start();
			},
			stop: function(){
				connection.stop();
			}
		}
	}
}




chrome.runtime.onConnect.addListener(function (port) {

     chrome.tabs.executeScript(parseInt(port.name),{ file: 'autodiscover.js' },function(result){

     	var options = result[0].split(";");
     	var settings = {};
     	for(var o in options){
     		var s = options[o].split('=');
     		settings[s[0]] = s[1];
     	}
     	
     	var lib = connectionlib[settings['library']]();
     	lib.init(settings,function(sql){
     		port.postMessage(sql);
     	});
     	
     });

   	  
      port.onDisconnect.addListener(function(p) {
    		lib.stop();
      });
 
});

The connectionlib object is just a simple way to handle support for multiple libraries. The listener function is where all the magic happens. For every dev tools panel which connects to it, it will attempt to detect if the inspected page supports real time logging and connect to it.

autodiscover.js
The background page will inject this code into the inspected window and if the it finds a meta tag with realtime logging configuration, it will send that configuration back to the background page.

var autoDiscover = document.querySelector('meta[name="real-time-log"][content]');
if(autoDiscover){
		autoDiscover.content + ';baseurl=' + window.location.protocol + '//'+ window.location.host
}

When I thought of ways the dev tools plugin could discover logging capabilities the first thing that came to my mind was meta tags. However, this can be achieved using custom headers or some other content in the page. Another option is to not use automatic discovery at all and opt for entering the url in the panel.

devtools.js
This code is very simple. All it does is create our logging panel when devtools opens.

chrome.devtools.panels.create("Real Time Log",
    "icon.png",
    "Panel.html",
    function(panel) {
      // code invoked on panel creation
    }
);

panel.js
This code will connect to the background page and wait for any incoming logs to output.

var log = document.getElementById('log');
var clear = document.getElementById('clear');

clear.addEventListener("click", function(){
	log.innerHTML = '';
});

var backgroundConnection = chrome.runtime.connect({
    name: ''+ chrome.devtools.inspectedWindow.tabId + ''
});

backgroundConnection.onMessage.addListener(function(sql){
	var li = document.createElement('pre');
	li.innerHTML =  hljs.highlight('sql', sql).value;
	log.appendChild(li);
});

panel.html
This page contains the elements the user can see an interact with in the devtools panel. The log element will display all log messages. Highlight will be used for syntax highlighting in the messages.

<html>
<head>
<link rel="stylesheet" href="lib/highlight/styles/xcode.css" />
<link rel="stylesheet" href="panel.css" />
</head>
<body>
<button id="clear">Clear</button>
<div id="log"></div>
<script src="lib/highlight/highlight.pack.js"></script>
<script src="panel.js"></script>
</body>
</html>

panel.css
This is some basic css for presenting the logs

pre {
	border-bottom:#cccccc 1px solid;
	padding-bottom:3px;
}

panel.css
This is some basic css for presenting the logs

pre {
	border-bottom:#cccccc 1px solid;
	padding-bottom:3px;
}

devtools.html
All this file does is include the devtools.js

<script src="devtools.js"></script>

What I have described so far in my two posts is really all you need for a basic implementation of this real time logging concept. You can download highlight.js from http://highlightjs.org/. I was only able to get the signalR client files by creating a dummy project and adding it to the project via nuget.

General Overview of the entire solution:
Real time plugin

The code in this post is a really basic get your hands dirty example. I created a github project which I will use to take the idea further. You are free to download the plugin, try it out and send pull requests if you wish. The project readme explains how to install and use the plugin.

Real time logs with Chrome dev tools and signalr part 1


This post was created to document the process of creating a Google Chrome Dev Tools extension which will allow a web application or plugin developer to get real time log information in the browser while developing. This first post will cover creating the web application which will log information to the dev tools extension. The second post will talk about creating the dev tools extension and connecting to the application.

What is this really about?
If you visit a site like bugs.mysql.com you will notice it tells you how long it took to generate the page. In my case it said this “Page generated in 0.017 sec. using MySQL 5.6.15-enterprise-commercial-advanced-log”. There are basically two types of logs, the ones that are persisted somehow on the server and others that are sent back to the client somehow. The mysql bugs page is an example of the latter. In this post I will be talking about the sending relevant information back to the client independent of any specific requests.

Ajax has changed everything
When I did constant WordPress development, there were many times my blog/app did not behave the way it was supposed to and I had no way of seeing what was going on. I eventually created a plugin which not only outputted all the request data, it also allowed me to output arbitrary logs, warnings, errors and sql statements together with the generated page. Fast forward to today where the apps I work on are about 90% asynchronous and views are handled on the client side, it is no longer convenient to simply output some arbitrary html/javascript to the bottom of every page. To solve this problem we need two things:

  • A way to transport the debug/log information to the client
  • A way to display that debug/log information on the client side once it is received

The first can be satisfied by making use of real time protocols such as WebSockets. This will continue to report back to the client even when a request fails. The second can be satisfied by creating a dev tools extension which will receive and display the debug/log information. Again this log lives in the browser and therefore will be independent of individual page requests.

A real use case
For the past couple years I have worked with asp.net mvc and entity framework quite a bit. Two common task I have are to figure out why certain records aren’t showing up on a given screen and why a given feature is slow. Part of my process is opening up sql profiler and logging any relevant sql queries which come in from the app. With this I can see whether or not the correct filters were applied via where clauses and also how long each individual query took to run. This works ok except that it is yet another window I need to open on my already crowded screen and it isn’t always easy to target the queries I am interested in. So what if instead of opening sql profiler, my sql statements came back to a neat little console in the browser where I am working? All I would have to do is open up dev tools and I would see all the sql activity as it happened. So in effect what I am looking for is a sql profiler but in the browser. One that only shows me relevant information.

Technology options
Before I go on, please note that my chosen technologies are strictly based on the fact that I develop mostly in asp.net mvc on sql server. However, this sort of thing can be done using Node.js and Socket.io or even Mono and XSockets.NET. So although I am doing this using SignalR, my proof of concept was actually done with Node.js and Socket.io.

Implementing the server side
The real time part of this is very simple because SignalR is really easy to setup and use in an application. You won’t even break a sweat adding it after the fact. For logging the sql statements, we will make use of the new interceptor api introduced in entity framework 6.

We will start off by creating a new asp.net mvc 5 project in visual studio
new app

Once the project has been created use the package manager console or the Nuget GUI to add the latest SignalR (2.0+ id:Microsoft.AspNet.SignalR), EntityFramework (6.0+) and jQuery(2.0+) to the project. Next create a new class which will act as the SignalR bootstrapper

[assembly: OwinStartup(typeof(RealTimeLogging.SignalRStartup))]
namespace RealTimeLogging
{
   
    public class SignalRStartup
    {
        public void Configuration(IAppBuilder app)
        {
            app.MapSignalR();
        }
    }

}

Next create the SignalR hub by creating a new class which extends Microsoft.AspNet.SignalR.Hub

namespace RealTimeLogging
{
    public class LoggingHub : Hub
    {
    }
}

The next class will allow us to send messages via any SignalR hub from anywhere in the application. This class can actually be used for any hub.

namespace RealTimeLogging
{
    public static class HubCaller
    {
        public static void Invoke<THub>(Action<IHubContext> action) where THub : IHub
        {
            var context = GlobalHost.ConnectionManager.GetHubContext<THub>();

            action.Invoke(context);
        }
    }
}

Next create a class which implements the IDbCommandInterceptor interface. This class will be used to intercept entity framework DbCommands and results and send the sql statements down to the client via our SignalR hub.

namespace RealTimeLogging
{
    public class StatementLogger : IDbCommandInterceptor
    {

        void IDbCommandInterceptor.NonQueryExecuted(System.Data.Common.DbCommand command, DbCommandInterceptionContext<int> interceptionContext)
        {
            SendToClient(command.CommandText);
        }

        void IDbCommandInterceptor.NonQueryExecuting(System.Data.Common.DbCommand command, DbCommandInterceptionContext<int> interceptionContext)
        {
            SendToClient(command.CommandText);
        }

        void IDbCommandInterceptor.ReaderExecuted(System.Data.Common.DbCommand command, DbCommandInterceptionContext<System.Data.Common.DbDataReader> interceptionContext)
        {
            SendToClient(command.CommandText);
        }

        void IDbCommandInterceptor.ReaderExecuting(System.Data.Common.DbCommand command, DbCommandInterceptionContext<System.Data.Common.DbDataReader> interceptionContext)
        {
            SendToClient(command.CommandText);
        }

        void IDbCommandInterceptor.ScalarExecuted(System.Data.Common.DbCommand command, DbCommandInterceptionContext<object> interceptionContext)
        {
            SendToClient(command.CommandText);
        }

        void IDbCommandInterceptor.ScalarExecuting(System.Data.Common.DbCommand command, DbCommandInterceptionContext<object> interceptionContext)
        {
            SendToClient(command.CommandText);
        }

        void SendToClient(string sql)
        {
            HubCaller.Invoke<LoggingHub>(_c => _c.Clients.All.onSql(sql));
        }
    }
}

The above interface gives us access to more than than just sql statements so the possibilities for adding to this class are endless. However, let us just keep it simple for now. Once registered with entity framework, the above class will send the command text of all DbCommands it receives to the client. There are several issues with the current implementation which we can fix later. The first is we assume all CommandText is sql. Another is we are sending messages to everyone instead of just the current user.

Next we will create an Entity Framework Code-First database and a client page to initiate queries so we have something to log.

Entity:

namespace RealTimeLogging
{
    public class Person
    {
        [DatabaseGenerated(DatabaseGeneratedOption.Identity)] 
        public int PersonID { get; set; }

        [StringLength(50)]
        public string FirstName { get; set; }

        [StringLength(50)]
        public string LastName { get; set; }

        public int Age { get; set; }
    }
}

Context:

namespace RealTimeLogging
{
    public class PersonContext : DbContext
    {
        public DbSet<Person> People { get; set; }
    }
}

Initializer:

namespace RealTimeLogging
{
    public class DbInitializer : DropCreateDatabaseAlways<PersonContext>
    {
        protected override void Seed(PersonContext context)
        {
            context.People.Add(new Person
            {
                FirstName = "John",
                LastName = "Doe",
                Age = 55

            });

            context.People.Add(new Person
            {
                FirstName = "Jane",
                LastName = "Smith",
                Age = 90

            });

            context.SaveChanges();
        }
    }
}

Db config where initializer and interceptor is registered with entity framework.

namespace RealTimeLogging
{
    public class DbConfig : DbConfiguration
    {
        public DbConfig()
        {
            this.SetDatabaseInitializer<PersonContext>(new DbInitializer());
            this.AddInterceptor(new StatementLogger());
        }
    }
}

Finally, the DbContext:

namespace RealTimeLogging
{
    [DbConfigurationType(typeof(DbConfig))]
    public class PersonContext : DbContext
    {
        public DbSet<Person> People { get; set; }
    }
}

Next add an empty MVC 5 controller
new controller

Create a new view for the index action in the controller:
Add view

In RouteConfig.cs change the default controller action from “Home” to “Person”.

namespace RealTimeLogging
{
    public class RouteConfig
    {
        public static void RegisterRoutes(RouteCollection routes)
        {
            routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

            routes.MapRoute(
                name: "Default",
                url: "{controller}/{action}/{id}",
                defaults: new { controller = "Person", action = "Index", id = UrlParameter.Optional }
            );
        }
    }
}

Next include the SingalR client script and a section for views to inject scripts at the bottom of _Layout.cshtml.

    <script src="~/Scripts/jquery-2.0.3.min.js"></script>
    <script src="~/Scripts/bootstrap.min.js"></script>
    <script src="~/Scripts/jquery.signalR-2.0.1.min.js"></script>
    @RenderSection("scripts",false)
</body>
</html>

At this point you should be able to run the mvc web application. You can find any missing “using” statements by right clicking classes and selecting the “Resolve” option. Once you have verified that the application can build we will create some controller actions and client side javascript which will interact with our database.

Replace the contents of Views/Person/Index.cshtml with the following:


@{
    ViewBag.Title = "Index";
}

<h2>Index</h2>
<button id="btnFirstPersonName">Name of First Person</button> <span id="FirstPersonName"></span><br /><br />
<button id="btnNumberOfPeople">Number of people</button> <span id="TotalPersons"></span>

@section scripts 
{
    <script type="text/javascript">
        $(function () {

            $('#btnFirstPersonName').click(function () {
                $.ajax({
                    url: '@Url.Action("FirstPersonName","Person")',
                    type: 'POST'
                }).done(function (data) {
                    $('#FirstPersonName').text(data);
                });
            });

            $('#btnNumberOfPeople').click(function () {
                $.ajax({
                    url: '@Url.Action("NumberOfPeople","Person")',
                    type: 'POST'
                }).done(function (data) {
                    $('#TotalPersons').text(data);
                });
           });

        });
    </script>

}

Change the PersonController class to look like the following:

    public class PersonController : Controller
    {
        PersonContext context = new PersonContext(); 
        //
        // GET: /Person/
        public ActionResult Index()
        {
            return View();
        }

        public JsonResult FirstPersonName()
        {
            var firstPerson = context.People.FirstOrDefault();
            return Json(firstPerson.FirstName + " " + firstPerson.LastName);
        }

        public JsonResult NumberOfPeople()
        {
            var numPeople = context.People.Count();
            return Json(numPeople);
        }
   }

At this point, if you run the app and press the two buttons your app should look like the following:
app working

Testing the Interceptor
Set a breakpoint inside the “SendToClient” method and click one of the buttons again. The app should stop at the breakpoint just like mine did. There will be several statements coming through here so you can keep going through to see the sort of commands Entity Framework sends to the database.

Logger debug

This concludes the first post which covered:

  • Creating a basic Asp.net MVC 5 application
  • Adding SignalR for transferring log data in real time
  • Entity Framework using Code First for our database
  • The new interceptor API for getting sql statements from Entity Framework
  • Setting up a basic page to call some controller actions which will query the database

The code used in this post can be found on Github. The next post will cover creating the dev tools extension which will display the log information.

C# datatables parser


The jQuery Datatables plugin is a very powerful javascript grid plugin which comes with the following features out of the box:

  • filtering
  • sorting
  • paging
  • jQuery ui themeroller support
  • plugins/extensions
  • Ajax/Remote and local datasource support

Setting up datatables on the client is very simple for basic scenarios. Here is an example of the markup and the initialization code.

<table id="PeopleListTable">
  <thead>
    <tr>
      <th>Name</th>
      <th>Age</th>
   </tr>
  </thead>
  <tbody>
    <tr>
      <td>John Doe</td>
      <th>25</th>
    </tr>
  </tbody>
</table>
$(function(){
  $('#PeopleListTable').dataTable();
});

Server Side Processing
The Datatables plugin supports loading table data, paging, sorting and filtering via ajax. Datatables sends a specific set of parameters which the server is expected to process and return the result in json format. Here is a sample of the request parameters sent via ajax:

sEcho:35
iColumns:7
sColumns:
iDisplayStart:0
iDisplayLength:10
mDataProp_0:FirstName
mDataProp_1:LastName
mDataProp_2:BirthDateFormatted
mDataProp_3:BirthDate
mDataProp_4:Weight
mDataProp_5:Height
mDataProp_6:Children
sSearch:
bRegex:false
sSearch_0:
bRegex_0:false
bSearchable_0:true
sSearch_1:
bRegex_1:false
bSearchable_1:true
sSearch_2:
bRegex_2:false
bSearchable_2:true
sSearch_3:
bRegex_3:false
bSearchable_3:true
sSearch_4:
bRegex_4:false
bSearchable_4:true
sSearch_5:
bRegex_5:false
bSearchable_5:true
sSearch_6:
bRegex_6:false
bSearchable_6:true
iSortCol_0:1
sSortDir_0:asc
iSortingCols:1
bSortable_0:true
bSortable_1:true
bSortable_2:true
bSortable_3:true
bSortable_4:true
bSortable_5:true
bSortable_6:true

For a detailed description of each parameter please see the datatables.net documentation

mDataProp_n Parameters
Datatables supports displaying columns in any order in the table by setting the mProp property of a column to a specific property in the json result array. For each column, it sends a parameter in the format ‘DataProp_columnIndex = propertyName’. As we can see in our example above, FirstName is the mProp of the first column in the table. It is important to understand these column index property mappings because the sorting and filtering parameters rely on them being interpreted properly.

Sorting
Datatables has a global setting called bSort which disables sorting for the entire table. It also has a property called bSortable which enables/disables sorting for a specific column. For each column, the server side script should search for a parameter in the format ‘bSortable_columnIndex = true/false’. Sorting is determined by parameters with the formats ‘iSortCol_sortCount = columnindex’ and ‘sSortDir_sortCount = asc’ where ‘sortCount’ is the order of sorted parameters and ‘asc’ is the direction that the specific column should be sorted.

Filtering
Datatables has a global setting called bFilter which disables filtering for the entire table. It also has a property called bSearchable which enables/disables filtering for a specific column. For each column, the server side script should search for a parameter in the format ‘bSearchable_columnIndex = true/false’. Filtering works by searching all the searchable columns in a row for any value which contains the filter value in the format ‘sSearch = findMe’. There is also support for filtering on specific columns by using the parameters in the format ‘sSearch_columnIndex = findMe’.

The c# Datatables Processor
The parser is a generic class with implements most of the server side features of the Datatables plugin in a reusable manner with special emphasis on performance. For example, an application which requires grids for people, cities and shopping lists does not require special logic for sorting and filtering each entity type because Datatables dynamically generates the expressions required to support these functions. If our first client side example was configured to use server side processing it would probably look like this:

 <table id="PeopleListTable"></table>
        $(function () {
            var peopleList = $('#PeopleListTable').dataTable({
                bServerSide: true,
                bProcessing: true,
                sServerMethod: "POST",
                sAjaxSource: "@Url.Action("All", "Person")",
                aoColumns: [
                    { mData: "FirstName", sTitle: "First Name" },
                    { mData: "LastName", sTitle: "Last Name"}
                ]
            });
        });
public JsonResult All()
{
    var context = new PeopleEntities();
    var parser = new DataTablesParser<Person>(Request, context.People);

    return Json(parser.Parse());
}

With the above combination of markup, javascript and 3 lines of server side code you have the ability to render a very rich and responsive grid in little time.

Entity Framework Performance
The parser supports two separate scenarios which are determined by the provider of the Iqueryable supplied to its constructor; The simple case where all/most processing is handled in memory via Linq to Objects and the more complex case where most/all processing is handled on the database server via Linq to SQL. In linq to sql support we ensure all the expressions sent to entity framework are translatable to valid tsql statements. The goal here is to avoid the cost of bringing most/all the data across the wire and into memory for processing. Imagine a grid for a dataset with 2 million records where you pull in all 2 million records from the database only to send 10 to the client.

As an example the following sql statement should be the result of the request it precedes. All the sorting, filtering and paging parameters have been translated and are represented in the tsql statement.

  SELECT TOP (10) [Filter1].[Id] AS [Id], 
                 [Filter1].[FirstName] AS [FirstName], 
                 [Filter1].[LastName] AS [LastName]
 FROM ( SELECT [Extent1].[Id] AS [Id], 
               [Extent1].[FirstName] AS [FirstName], 
               [Extent1].[LastName] AS [LastName], 
               row_number() OVER (ORDER BY [Extent1].[FirstName] ASC) AS [row_number]
               FROM [dbo].[People] AS [Extent1]
               WHERE ([Extent1].[FirstName] LIKE N'%john%') 
                     OR ([Extent1].[LastName] LIKE N'%john%')\r\n)  AS [Filter1]
 WHERE [Filter1].[row_number] > 0
 ORDER BY [Filter1].[FirstName] ASC
sEcho:35
iColumns:7
sColumns:
iDisplayStart:0
iDisplayLength:10
mDataProp_0:FirstName
mDataProp_1:LastName
sSearch: john
iSortCol_0:1
sSortDir_0:asc

The ‘iDisplayStart’ property determines the start of a page of data and iDisplayLength determines the length of each page of data.

Where is X feature?
The biggest feature missing from the parser is processing individual search filters. Originally, the individual property search and the generic search were implemented as two separate functions. However, I am convinced that the bulk of the logic in the generic search can be generalized to also handle the individual property such. I am open to any ideas on this one. I have also been asked about sorting/filtering on sub properties. This should be possible in linq to objects but I have not been able to look into it.

Conclusion
The parser is definitely a work in progress in the sense that it is always being improved whenever possible but it certainly saves time when using the datatables plugin for grids.

The parser can be added to your project via npm using the following command:

PM> Install-Package DataTablesParser

Please note that the NPM version up until the publishing of this post does not have the most up to date fixes and changes. I plan to update the NPM package as soon as the new changes have been thoroughly tested.

You can get the latest code or send pull requests at the github repository here:
https://github.com/garvincasimir/csharp-datatables-parser

jQueryUi Dialog Search using knockoutjs


This is a followup to a post I wrote about using a jQueryUi dialog as a popup search with a filter. I decided to convert the example to use knockoutjs for the data bindings. The following is what I came up with.

Markup:

</pre>
<div id="SearchDialog">
<div class="SearchTitle">
<div class="SearchTitleText">Search Friends</div>
<div class="SearchTitleFilterButton">Filter:
 <input class="SearchFilter" type="text" data-bind='value: filter, valueUpdate: "afterkeydown"' />
 <button class="SearchButtonClose"></button></div>
<div class="SearchClear"></div>
</div>
<div class="SearchContent" data-bind="foreach: filteredFriends"></div>
<div data-bind="visible: filteredFriends().length == 0">No Results</div>
</div>
<pre>

In the above note the new data-bind attributes which knockoutjs uses to perform its magic. It now handles both updating of the DOM and the keyboard events which trigger the filtering.

The Javascript:


        $(function(){
            $('#SearchDialog').dialog({
                    width: 500,
                    create: function () {
                        //Get a reference to jquery ui generated element

                        var widget = $(this).dialog('widget');
                        //Get a reference to content div
                        var dialog = $(this);
                        //Delete the default jquery ui title bar
                        widget.find('.ui-dialog-titlebar').remove();

                        //Remove the default padding values for the popup
                        widget.css('padding', 0).find('.ui-dialog-content').css('padding', 0);

                        //Use jquery ui styling for our new titlebar
                        $('.SearchTitle',dialog).addClass('ui-widget-header');

                        //Position title elements
                        $('.SearchTitleText',dialog).css('float','left');
                        $('.SearchTitleFilterButton',dialog).css('float','right');
                        $('.SearchClear',dialog).css('clear','both');

                        //Create a close button and allow it to close the popup
                        $('.SearchButtonClose',dialog).button({
                            icons: { primary: 'ui-icon-close' },
                            text: false
                        }).css('width', 16)

                        .click(function () {
                            dialog.dialog('close');
                        });

                    }

            });

            var friendsModel = function(){
                var self = this;

                self.friends = ko.observableArray([]);
                self.filter = ko.observable("");
                self.select = function(n){
                    alert('Hello I am ' + n);
                };

                self.filteredFriends = ko.computed(function() {
                	var filterText = self.filter().toLowerCase();
                	return ko.utils.arrayFilter(self.friends(), function(item) {
                   			return filterText.length < 1 || item.toLowerCase().indexOf(filterText)!=-1
               	    });

               }, self);

                //Load Json when model is setup
                self.dummyCompute = ko.computed(function() {
                    $.getJSON("friends", function(data) {
                    		self.friends(data);
                 	 });

                }, self);
            };

            ko.applyBindings(new friendsModel());

        });

In this example I decided to show how the data can be populated using ajax so I added a special computed observable which has the effect of loading the json array once the model is bound. Here is the java servlet code which returns the json data.


public class FriendList extends HttpServlet {
	public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
		String[] friends = new String[] {"John","Bob","Jones","Tessa","Anna","Casper","Ed","Francis","Greg","Hanna","Ivan"};
		Gson gson = new Gson();
		String jsonString = gson.toJson(friends);
		resp.setContentType("application/json");
		resp.getWriter().print(jsonString);
	}

}

If you are wondering why I am using java it is because my samples are hosted in google app engine. The live example for this post can be found here. Also, the non ajax example using knockoutjs can be found here.

Tutorial: Mvc application using Azure acs and forms authentication Part 2


This is the second post in a series of posts on converting an asp.net mvc project to use both forms authentication and Azure appFabric Access Control Service (ACS) authentication. The first post focused on creating the project, configuring the site for acs and forms authentication and setting up the database. This post will focus on the association process and the hybrid form where users can choose between forms authentication and ACS authentication.

Important code from ACS Mvc3 sample

Next you will need some code from the MVC3CustomSignInPage sample. Download the source from acs.codeplex.com (please note that these samples are under the Apache 2 license). We will be making changes to some of these files but I thought it would be good for people to know their origin. Extract the package and browse to the Websites\MVC3CustomSignInPage\MVC3CustomSignInPage directory and do the following.

  • Copy the Hrd folder to your project
  • Copy the Util folder to your project
  • Copy Models\HrdIdentityProvider.cs to the Models folder in your project
  • Open Controllers\AccountController.cs and copy the GetUrlFromContext method to your AccountController.cs
  • Copy Views\Account\_IdentityProvidersWithServerSideCode.cshtml to Views\Account in you project.
  • Copy Content\HrdPage.css to Content in your project

Now we should have all the code we need to complete the project.

Hybrid logon page

You must set the request validation mode in the web.config else you will get an error upon returning from ACS. Open web.config and find the following:

<authentication mode="Forms"><forms loginUrl="~/Account/LogOn" timeout="2880" /></authentication>

Below insert the following

<httpRuntime requestValidationMode="2.0" />

Add a section to the layout for any custom styles we will render in our views. And modify our menu to include a link to a list of associated identities. Open Views\Shared\_Layout.cshtml add replace the code with the following:

<!DOCTYPE html>
<html>
<head>
    <title>@ViewBag.Title</title>
    <link href="@Url.Content("~/Content/Site.css")" rel="stylesheet" type="text/css" />
    @RenderSection("Styles",false)
    <script src="@Url.Content("~/Scripts/jquery-1.5.1.min.js")" type="text/javascript"></script>
</head>
<body>
    <div class="page">
        <div id="header">
            <div id="title">
                <h1>My MVC Application</h1>
            </div>
            <div id="logindisplay">
                @Html.Partial("_LogOnPartial")
            </div>
            <div id="menucontainer">
                <ul id="menu">
                    <li>@Html.ActionLink("Home", "Index", "Home")</li>
                    <li>@Html.ActionLink("Identities", "Identities", "Account")</li>
                </ul>
            </div>
        </div>
        <div id="main">
            @RenderBody()
        </div>
        <div id="footer">
        </div>
    </div>
</body>
</html>

Open Views\Account\LogOn.cshtml and replace the code with the following:

@using ASPNETSimpleMVC3.Models;
@model MVC3MixedAuthenticationSample.Models.LogOnModel

@{
    ViewBag.Title = "Log On";
}

@section Styles {
    <link href="@Url.Content("~/Content/HrdPage.css")" rel="stylesheet" type="text/css" />
    <style type="text/css">
        #FormsAuth 
        {
            width:400px;
            margin:0;
            padding:0;
            float:left;
        }
    </style>
}



<h2>Log On</h2>
<p>
    Please enter your user name and password. @Html.ActionLink("Register", "Register") if you don't have an account.
</p>

@*Needed for Visual Studio Intellisense*@
@if (false)
{
    <script src="/Scripts/jquery-1.5.1-vsdoc.js" type="text/javascript"></script>
}

<script src="@Url.Content("~/Scripts/jquery.validate.min.js")" type="text/javascript"></script>
<script src="@Url.Content("~/Scripts/jquery.validate.unobtrusive.min.js")" type="text/javascript"></script>

<script type="text/javascript">
    $(function () {
        $('#AssociateCancel').click(CancelAssociation);
    });

    function CancelAssociation() {
        $.post('@Url.Action("CancelAssociation")',function(data){
           if(data) {
          
            $('#identity-provider-content').show();
            $('#AssociationMessage').hide();
           } 
           else{
            alert("Unexpected error. Try again later");
           }
        });
        
    }
    
</script>


@Html.ValidationSummary(true, "Login was unsuccessful. Please correct the errors and try again.")
@if (ViewData["HasClaim"]!=null)
{
<div id="AssociationMessage">
    <div id="AssociateMessageText">Please login to associate your account</div>
    <div id="AssociateMessageActions"><button id="AssociateCancel">Cancel</button></div> 
    <div class="clear"></div>
</div>
}
@using (Html.BeginForm()) {
    <div id="FormsAuth">
        <fieldset>
            <legend>Account Information</legend>

            <div class="editor-label">
                @Html.LabelFor(m => m.UserName)
            </div>
            <div class="editor-field">
                @Html.TextBoxFor(m => m.UserName)
                @Html.ValidationMessageFor(m => m.UserName)
            </div>

            <div class="editor-label">
                @Html.LabelFor(m => m.Password)
            </div>
            <div class="editor-field">
                @Html.PasswordFor(m => m.Password)
                @Html.ValidationMessageFor(m => m.Password)
            </div>

            <div class="editor-label">
                @Html.CheckBoxFor(m => m.RememberMe)
                @Html.LabelFor(m => m.RememberMe)
            </div>

            <p>
                <input type="submit" value="Log On" />
            </p>
        </fieldset>

    </div>
    
}
@Html.Partial("_IdentityProvidersWithServerSideCode")

Open Views\Account\_IdentityProvidersWithServerSideCode.cshtml and replace the code with the following:

@using ASPNETSimpleMVC3.Models;
@{IEnumerable<HrdIdentityProvider> Providers = null;}
@{
    if (!string.IsNullOrEmpty(ViewData["Providers"].ToString()) && (ViewData["Providers"] as IEnumerable<HrdIdentityProvider>).Count() > 0)
    { 
        Providers = ViewData["Providers"] as IEnumerable<HrdIdentityProvider>;
    }
}
  

@*Needed for Visual Studio Intellisense*@
@if (false)
{
    <script src="/Scripts/jquery-1.5.1-vsdoc.js" type="text/javascript"></script>
}

@if (Providers != null)
{
<div id="identity-provider-content"  class="@(ViewData["HasClaim"] != null ? "provider-content-hidden" : "")">
    
    
    <h2>
        Or sign in using
    </h2>
    <div id="identity-providers">
        @foreach (HrdIdentityProvider identityProvider in Providers)
        {
            <div  class="identity-provider"><button onclick="window.location='@Html.Raw(identityProvider.LoginUrl)'">@identityProvider.Name</button></div>
        }
    </div>
</div>

<div class="clear"></div>
}

Open Content/HrdPage.css and change the code to the following:


#identity-provider-content
{
    width:250px;
    text-align:center;   
    float:left;
   
}

#identity-providers
{
    text-align:center;
    width:100%;
}

div.identity-provider
{

    text-align:center;
    margin: 10px 0px 10px 0px;
}

div.identity-provider button
{
    padding:10px;
    width:150px;
    vertical-align:middle;
}

#AssociationMessage
{
    border: 1px solid #4F8A10;
    background-color: #DFF2BF;
    color:#4F8A10;
    padding:5px;
    width:400px;
}

.provider-content-hidden
{
    display:none;
}

#AssociateMessageActions
{
    float:right;
}
#AssociateMessageText
{
    float:left;
}

Open Controllers\AccountController.cs and make the following changes:

Add an action called SignIn

        /// <summary>
        /// Action used to sign user in from ACS
        /// </summary>
        /// <param name="forms"></param>
        /// <returns></returns>
        [HttpPost]
        [ValidateInput(false)]
        public ActionResult SignIn(FormCollection forms)
        {
            //Extract claims
            var principal = HttpContext.User;
            var claim = new IdentityClaim(principal.Identity as IClaimsIdentity);

            //Delete session cookie so the module cannot reset principal
            SessionAuthenticationModule sam = FederatedAuthentication.SessionAuthenticationModule;
            sam.DeleteSessionTokenCookie();

            if (claim.HasIdentity)
            {
                var db = new IdentityRepository();

                var identity = db.FindByProvderAndValue(claim.IdentityProvider, claim.IdentityValue);
                string returnUrl = GetUrlFromContext(forms) ?? Url.Action("Index", "Home", null);
                if (identity != null)
                {
                    var user = Membership.GetUser(identity.UserId);

                    if (user != null)
                    {
                        FormsAuthentication.SetAuthCookie(user.UserName, false);
                        return Redirect(returnUrl);
                    }

                }
                else
                {
                    //Save identity values for processing in the association page
                    claim.SaveToSession();

                    return RedirectToAction("LogOn", new { ReturnUrl = returnUrl });

                }

            }

            return RedirectToAction("LogOn", "Account");
        }

Still in Controllers\AccountController.cs add the following actions which will facilitate the association process.

         /// <summary>
        /// Remove pending association from session
        /// </summary>
        /// <returns></returns>
        [Authorize]
        public JsonResult CancelAssociation()
        {
            IdentityClaim.ClearSession();

            return Json(true);
        }

        /// <summary>
        /// Allows a user to remove an identity association
        /// </summary>
        /// <param name="id"></param>
        /// <returns></returns>
        [Authorize]
        public JsonResult RemoveAssociation(int id)
        {
            var db = new IdentityRepository();
            var user = Membership.GetUser();
            db.DeleteById(id,user.ProviderUserKey.ToString());

            return Json(true);
        }

        /// <summary>
        /// Allows users to manage their associated identites
        /// </summary>
        /// <returns></returns>
        [Authorize]
        public ActionResult Identities()
        {
            var db = new IdentityRepository();
            var user = Membership.GetUser();

            var identities = db.FindByUserId(user.ProviderUserKey.ToString());

            return View(identities);
        }

Add a new view to Views\Account called Identities.cshtml and insert the following code:

@model IEnumerable<MVC3MixedAuthenticationSample.UserIdentity>
@using MVC3MixedAuthenticationSample.Models

@{
    ViewBag.Title = "Identities";
}

@*Needed for Visual Studio Intellisense*@
@if (false)
{
    <script src="/Scripts/jquery-1.5.1-vsdoc.js" type="text/javascript"></script>
}

<script type="text/javascript">
    function RemoveAssociation(element, id) {
        $.post('@Url.Action("RemoveAssociation")', { id: id }, function (data) {
            if (data) {
                $(element).parent().slideUp().Remove();
            }
            else {
                alert("There was an unexpected error. Try again later");
            }
        });
    }
</script>
<h2>Identities</h2>

<div id="ProviderList">
@foreach (var item in Model) {
  <div>
    @IdentityClaim.ProviderNiceName( item.IdentityProvider )  <button onclick="RemoveAssociation(this,@item.IdentityID)">Remove</button>
  </div>
}
</div>



Replace the parameter-less LogOn() action with the following code:

        public ActionResult LogOn()
        {
            HrdClient hrdClient = new HrdClient();

            WSFederationAuthenticationModule fam = FederatedAuthentication.WSFederationAuthenticationModule;
            HrdRequest request = new HrdRequest(fam.Issuer, fam.Realm, context: Request.QueryString["ReturnUrl"]);

            IEnumerable<HrdIdentityProvider> hrdIdentityProviders = hrdClient.GetHrdResponse(request);

            var claim = new IdentityClaim();
            if (claim.HasIdentity)
            {
                ViewData["HasClaim"] = true;
            }

            ViewData["Providers"] = hrdIdentityProviders;
            return View();
        }

Replace the LogOn(LogOnModel model, string returnUrl) action with the following code:

        [HttpPost]
        public ActionResult LogOn(LogOnModel model, string returnUrl)
        {
            var claim = new IdentityClaim();

            if (ModelState.IsValid)
            {
                if (Membership.ValidateUser(model.UserName, model.Password))
                {
                   

                    if (claim.HasIdentity)
                    {
                        var db = new IdentityRepository();
                        var user = Membership.GetUser(model.UserName);
                        db.MapIdentity(user.ProviderUserKey.ToString(), claim.IdentityProvider, claim.IdentityValue);
                        IdentityClaim.ClearSession();
                    }

                    FormsAuthentication.SetAuthCookie(model.UserName, model.RememberMe);
                    if (Url.IsLocalUrl(returnUrl) && returnUrl.Length > 1 && returnUrl.StartsWith("/")
                        && !returnUrl.StartsWith("//") && !returnUrl.StartsWith("/\\"))
                    {
                        return Redirect(returnUrl);
                    }
                    else
                    {
                        return RedirectToAction("Index", "Home");
                    }
                }
                else
                {
                    ModelState.AddModelError("", "The user name or password provided is incorrect.");
                }
            }

            HrdClient hrdClient = new HrdClient();

            WSFederationAuthenticationModule fam = FederatedAuthentication.WSFederationAuthenticationModule;
            HrdRequest request = new HrdRequest(fam.Issuer, fam.Realm, context: Request.QueryString["ReturnUrl"]);

            IEnumerable<HrdIdentityProvider> hrdIdentityProviders = hrdClient.GetHrdResponse(request);

            if (claim.HasIdentity)
            {
                ViewData["HasClaim"] = true;
            }

            ViewData["Providers"] = hrdIdentityProviders;
            // If we got this far, something failed, redisplay form
            return View(model);
        }

Next open controllers\HomeController.cs and add the authorize attribute so we can test our new feature.

    [Authorize]
    public class HomeController : Controller
    {
      ...
    }

Add a new class called IdentityRepository.cs to the models folder and insert the following code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace MVC3MixedAuthenticationSample.Models
{
     
    public class IdentityRepository 
    {
        private ASPNETDBEntities _context;

        public IdentityRepository()
        {
            _context = new ASPNETDBEntities();
        }

        public int MapIdentity(string userid, string identytProvider, string identityValue)
        {
            var mapping = new UserIdentity();
            mapping.UserId = new Guid(userid);
            mapping.IdentityProvider = identytProvider;
            mapping.IdentityValue = identityValue;
            _context.AddToUserIdentities(mapping);
            _context.SaveChanges();

            return mapping.IdentityID;
        }

        public IQueryable<UserIdentity> FindByUserId(string userId)
        {
           return _context.UserIdentities.Where(i => i.UserId == new Guid(userId));
        }

        public UserIdentity FindByProvderAndValue(string IdentityProvider, string identityValue)
        {
            return _context.UserIdentities.FirstOrDefault(i => i.IdentityProvider == IdentityProvider && i.IdentityValue == identityValue);
        }

        public void DeleteById(int id, string userId)
        {
            var identity = _context.UserIdentities.FirstOrDefault(i => i.IdentityID == id && i.UserId == new Guid(userId));
            if (identity != null)
            {
                _context.DeleteObject(identity);
            }
            _context.SaveChanges();
        }

       
    }
}

At this point the solution should look like this:

Next run the project and verify that the LogOn page looks like the following:

At this point you must verify that ACS is setup properly with the project url as a valid relying party (Please see the original ACS MVC3 sample tutorial if you have not been able to set this up) and has http://%5Bprojecturl%5D/Account/SignIn as the return url. If this is not configured properly you will get a bad request exception when attempting to load the Login page.

If you have not already selected the providers you want users to authenticate against you should do so at this point. Again, there are many tutorials out there which cover this process. Facebook is the only provider which requires extra steps to be added as a provider. You can read about it here.

You should now be able to click on one of the provider buttons on the LogOn page where you will sign in using any of the providers setup in your ACS account. After signing in you will return to the following screen on your site which prompts you to login to complete the association process:

After you login you should see your new identity listed on the Identites page.

Finally, you should be able to logout then click the same provider button you clicked before to get logged in without having to enter your username and password for the site.

The sample project used in this tutorial is available on codeplex for anyone who wants it. Have fun with ACS and Asp.net memberships.

To use the sample project you will need to configure the Windows Identity framework module for your own ACS namespace.

Dotnetnuke Ajax WebLinks module on Codeplex


I promised to release the complete code for the Dotnetnuke Ajax WebLinks module I have been working on. It was supposed to demonstrate a method for including jquery ajax functionality in dotnetnuke modules. However, I have not had time to finish it and write up the last two posts. In the mean time I posted the project on codeplex so people can view all the nearly complete source code. When I get some time soon I will finish this one and possibly create a version for Dotnetnuke 6.x if there isn’t already something built in that makes it unnecessary.

It can be found here

jQueryUi Dialog Search


I use the jQueryUI dialog widget quite extensively in my ajax enabled web projects. Recently, I decided to use it as a popup search box by adding a text input to the title bar. Let’s call it jQueryUi Dialog Search.

I opened up my trusty OmniGraffle and came up with the following:

The following is the markup for the custom dialog box featuring my new custom titlebar


<div id="SearchDialog">
<div class="SearchTitle" >
<div class="SearchTitleText">Search Friends</div>
<div class="SearchTitleFilterButton">
<span>Filter:</span>
<input type="text" class="SearchFilter" />
<button class="SearchButtonClose" ></button>
</div>
<div class="SearchClear"></div>
</div>
<div class="SearchContent"> </div>
</div>

I used the “create” event to remove the default title bar and setup my custom title

$('#SearchDialog').dialog({
width: 500,
create: function () {
//Get a reference to jquery ui generated element
var widget = $(this).dialog('widget');

//Get a reference to content div
var dialog = $(this);

//Delete the default jquery ui title bar
widget.find('.ui-dialog-titlebar').remove();

//Remove the default padding values for the popup
widget.css('padding', 0).find('.ui-dialog-content').css('padding', 0);

//Use jquery ui styling for our new titlebar
$('.SearchTitle',dialog).addClass('ui-widget-header');

//Position title elements
$('.SearchTitleText',dialog).css('float','left');
$('.SearchTitleFilterButton',dialog).css('float','right');
$('.SearchClear',dialog).css('clear','both');

//Create an event for the search input
$('.SearchFilter',dialog).keyup(function(){

ShowResults($(this).val(),friends);
});

//Create a close button and allow it to close the popup
$('.SearchButtonClose',dialog).button({
icons: { primary: 'ui-icon-close' },
text: false
}).css('width', 16)
.click(function () {
dialog.dialog('close');
});

//Show all results on create
ShowResults('',friends);

}

});

Now to complete this example you will need a datasource, way to search for items and render the results

var friends = ["John","Bob","Jones","Tessa","Anna","Casper","Ed","Francis","Greg","Hanna","Ivan" ];
//Show results in search window based on input<
function ShowResults(filterText,dataSource)
{
var html = ''

$.each(dataSource,function(index,value){
if(filterText.length < 1 || value.toLowerCase().indexOf(filterText)!=-1){
html += '<li><a href="#" onClick="SelectResult(this)">'+value+'</a></li>';
}

});

$('#searchDialog .SearchContent').html(html==''?'No Results':'<ul>'+html+'</ul>');

}

//Use this function to select an item
function SelectResult(element){
alert('Hello, I am ' + $(element).text())
}

The end result looks like this:

You can also try the live demo in my app engine playground.

I hope this will be helpful to someone. Cheers!