Elasticsearch In Azure PAAS [Worker Roles]


The Preamble
A few months ago I came across the free law project and its awesome dataset of court decisions from around the country. I think it is amazing that everyday people can have easy and free access to a searchable index of court decisions. In addition to their front end search page and search api, they allow more advanced people to download their dataset using their bulk api. People are doing some pretty interesting things with this dataset. I even came across what looked like a decision prediction engine. I decided to use the dataset to create my own search index because it was the perfect opportunity to play around with Elasticsearch (a distributed search index). Note that I have no intention of replacing the existing functionality provided on courtlistener.com. This is more of an academic exercise for me.

Getting and processing the data
The bulk api outputs a compressed file containing opinions in xml format. I started off by creating a bash script which accepts one of the compressed files outputted by the bulk api, iterates over the opinions and get them ready for import into the search index. Creating the script proved challenging at first because I did not have enough disk space available on my MBP to store and processed the 12gb file. I then realized that i have tons of space available to me in Azure storage so I created an Azure ubuntu vm with an attached disk to test my processing script.

Choosing a platform
By the time I was done with the script I started thinking about how I would get Elasticsearch stood up on multiple nodes with discovery enabled. I came across the Elasticsearch Cloud Azure Plugin which is maintained by Elasticsearch. This plugin uses the Azure Management Api for discovery and the Azure Storage api for snapshots. I started implementing it but then I thought maybe it didn’t have to be this hard. Why should I have to be messing with certificate stores and all that when the service runtime api provides all the info I need for discovery. This is when I decided to focus my efforts on getting this going in the Azure PAAS environment. I came up with a plan of attack to get things going:

  • Get Java Installed
  • Persistent Storage for data
  • Persistent Storage for snapshots
  • Configuring and running Elasticsearch
  • A discovery plugin based on the runtime api
  • Internal load balanced endpoints
  • Logging

At this point I was confident I could get everything done using Azure Worker roles.

Elasticsearch PAAS

Setting up the project
Getting Java installed was a no brainer. All I needed to do was create a startup script which downloaded and run the installer. After a few bruises and a couple of hours into creating the startup script as a batch file, I decided it didn’t make sense to be battling with batch scripts when I had access to the full .net framework and a much more modern scripting engine in powershell. I also decided that I should include the java installer in the project instead of downloading it because it made more sense to take that hit when uploading the package instead of during startup. Once I had a working powershell script which ensured java was installed, all I needed to do was configure and run Elasticsearch.

Elastic PAAS Solution

Persistent Storage
Azure PAAS is a mostly stateless environment so applications must point to some external persistent storage. I was thinking that I could attach a vhd and mount it on my worker instances but eventually decided to try the new Azure File service with SMB support. It turns out that someone had already thought of this and released some code which made it very easy to mount Azure file storage shares as mapped drives in worker roles. With that, my persisted storage problem was solved. Elasticsearch could store its data on what would look like just another local drive. It is interesting to note that this is the only aspect of this solution I could not test on the Azure Emulator. So the code uses resource directories for storage when in the emulator.

Configuring and Running Elasticsearch
Elasticsearch does not have an installer per se. Instead, you use platform specific scripts to run it or install it as a service. So all I needed to do in the RoleEntryPoint was start the process and wait for it to exit. Also, when then RoleEntryPoint exists I can have it stop the process. The Elasticsearch configuration can be placed in either a .yml or .json file. I didn’t find out about the .json file till after so I pulled in a project called YamlDotNet to programmatically write configuration values such as the node name (instance id) that would only be available in the RoleEntryPoint. The deployment starts off with a base .yml, merges the base config with its own custom values in memory and writes the merged file to the final Elasticsearch config directory.

A discovery plugin based on the runtime api
I thought the discovery plugin would have been a walk in the park until I encountered a big problem. The Runtime API for java is supported by named pipes. Unfortunately, the named pipe is only available when using a ProgramEntryPoint in the service definition (I wish they would change that or at least make it configurable in the service definition). At this point I thought of moving the entire project to eclipse or using a console application as a ProgramEntryPoint but that took away my ability to simply run and debug in the emulator. Then, I thought if they were already using IPC for the java service runtime then it should be good enough for my solution. All I had to do was setup a mini server in the RoleEntryPoint that the java discovery plugin could communicate with. I did a proof of concept for both TCP and named pipes but eventually decided on named pipes because it was simpler. So with a working named pipes server answering all the questions about the state of the cloud service and the persistent storage for both data and snapshots abstracted away I was able to remove the dependency on the azure api for java in my plugin project.

Internal load balanced endpoints
My goal for this solution is to not expose the Elasticsearch cluster publicly but rather create a WebRole with a custom ui and or another WorkerRole which exposes a custom api to interact with the elasticsearch index. Looking at the service definition and reading around the internet it seems that I can’t declare an internal load balanced endpoint which can only be accessed by my public facing roles. It seems like my only option is to write a thin layer which cycles through or returns random node endpoints from the Elasticsearch cluster.

Logging
Elasticsearch is currently logging to a resource folder on the role, but I would really like to ship the logs somewhere. I am thinking that I can somehow hookup Elasticsearch logging to the diagnostic trace store which gets shipped to Azure storage or simply mount another Azure File share dedicated to logging. I also plan to read up on Logstash but I am not sure what value it would provide to my solution.

Additional Thoughts
Once I have all the configuration pieces in place, publishing this service will be super easy. With the click of a button (or two) I would have a private Elasticsearch cluster and a public facing website/api for interacting with it. I already have a script which will allow me to bootstrap the data but I have not thought of how to integrate it into the deployment workflow. The other challenge is scaling Elasticsearch. Scaling Elasticsearch is not as straight forward as adding more identical nodes to something such as a web application. You really have to put some thought into it. For this project I am thinking that the data, once bootstrapped, would be mostly read only. Therefore, the requirements will differ from other clusters. For example, I can have a set of main data nodes on one WorkerRole and a set of replica only nodes (need to read up on this) in another WorkerRole configured with autoscaling.

Conclusion
Thanks for taking the time to read my random stuff. The code for the project inspired by this proof of concept can be found on Github: https://github.com/garvincasimir/Elasticsearch-Azure-PAAS

Advertisements

Tutorial: Mvc application using Azure acs and forms authentication Part 3


This is the final post in a series of posts on converting an asp.net mvc project to use both forms authentication and Azure appFabric Access Control Service (ACS) authentication. The first post focused on creating the project, configuring the site for acs and forms authentication and setting up the database. This second poste focused on the association process and the hybrid form where users can choose between forms authentication and ACS authentication. This final post will add some more steps to the original workflow which will cover user signup.

This simplifies the process for non-members who would otherwise have to signup then logout to begin the association process.

Original workflow:

New Workflow:

Open the MVC3MixedAuthenticationSample from the two previous posts and make the following changes:

Open Controllers\AccountController.cs and replace the Register(RegisterModel model) action with the following:

        [HttpPost]
        public ActionResult Register(RegisterModel model)
        {
            if (ModelState.IsValid)
            {
                // Attempt to register the user
                MembershipCreateStatus createStatus;
                Membership.CreateUser(model.UserName, model.Password, model.Email, null, null, true, null, out createStatus);

                if (createStatus == MembershipCreateStatus.Success)
                {
                    var claim = new IdentityClaim();
                    if (claim.HasIdentity)
                    {
                        var db = new IdentityRepository();
                        var user = Membership.GetUser(model.UserName);
                        db.MapIdentity(user.ProviderUserKey.ToString(), claim.IdentityProvider, claim.IdentityValue);
                        IdentityClaim.ClearSession();
                    }

                    FormsAuthentication.SetAuthCookie(model.UserName, false /* createPersistentCookie */);
                    return RedirectToAction("Index", "Home");
                }
                else
                {
                    ModelState.AddModelError("", ErrorCodeToString(createStatus));
                }
            }

            // If we got this far, something failed, redisplay form
            return View(model);
        }

Open Views\Account\LogOn.cshtml and find the following:

 $('#AssociateCancel').click(CancelAssociation);

Immediately below insert the following code:

         $('#AssociateRegister').click(function () {
            window.location = '@Url.Action("Register")';
        });

Next Find the following code:

<div id="AssociationMessage">
    <div id="AssociateMessageText">Please login to associate your account</div>
    <div id="AssociateMessageActions"><button id="AssociateCancel">Cancel</button></div> 
    <div class="clear"></div>
</div>

Immediately below insert the following code:

 <div id="AssociationMessageRegister">
    <div id="AssociateMessageRegisterText">Click Register if you don't already have an account</div>
    <div id="AssociateMessageRegisterActions"><button id="AssociateRegister">Register</button></div> 
    <div class="clear"></div>
</div>

Open Content\HrdPage.css and insert the following css:

#AssociateMessageRegisterActions
{
    float:right;
}

#AssociateMessageRegisterText
{
    float:left;
}

#AssociationMessageRegister
{
    border: 1px solid #9F6000;
    background-color: #FEEFB3;
    color:#9F6000;
    padding:5px;
    width:400px;
}

Next run the site and click one of the provider buttons and login with a provider account that is not already associated. You can also go to Account/Identites and remove any existing associations if you don’t have another one to spare.

When you return to the site your login page should look like this:

Register as normal:

Once the user completes registration they will be able to use their identity provider to access the site.

Tutorial: Mvc application using Azure acs and forms authentication Part 2


This is the second post in a series of posts on converting an asp.net mvc project to use both forms authentication and Azure appFabric Access Control Service (ACS) authentication. The first post focused on creating the project, configuring the site for acs and forms authentication and setting up the database. This post will focus on the association process and the hybrid form where users can choose between forms authentication and ACS authentication.

Important code from ACS Mvc3 sample

Next you will need some code from the MVC3CustomSignInPage sample. Download the source from acs.codeplex.com (please note that these samples are under the Apache 2 license). We will be making changes to some of these files but I thought it would be good for people to know their origin. Extract the package and browse to the Websites\MVC3CustomSignInPage\MVC3CustomSignInPage directory and do the following.

  • Copy the Hrd folder to your project
  • Copy the Util folder to your project
  • Copy Models\HrdIdentityProvider.cs to the Models folder in your project
  • Open Controllers\AccountController.cs and copy the GetUrlFromContext method to your AccountController.cs
  • Copy Views\Account\_IdentityProvidersWithServerSideCode.cshtml to Views\Account in you project.
  • Copy Content\HrdPage.css to Content in your project

Now we should have all the code we need to complete the project.

Hybrid logon page

You must set the request validation mode in the web.config else you will get an error upon returning from ACS. Open web.config and find the following:

<authentication mode="Forms"><forms loginUrl="~/Account/LogOn" timeout="2880" /></authentication>

Below insert the following

<httpRuntime requestValidationMode="2.0" />

Add a section to the layout for any custom styles we will render in our views. And modify our menu to include a link to a list of associated identities. Open Views\Shared\_Layout.cshtml add replace the code with the following:

<!DOCTYPE html>
<html>
<head>
    <title>@ViewBag.Title</title>
    <link href="@Url.Content("~/Content/Site.css")" rel="stylesheet" type="text/css" />
    @RenderSection("Styles",false)
    <script src="@Url.Content("~/Scripts/jquery-1.5.1.min.js")" type="text/javascript"></script>
</head>
<body>
    <div class="page">
        <div id="header">
            <div id="title">
                <h1>My MVC Application</h1>
            </div>
            <div id="logindisplay">
                @Html.Partial("_LogOnPartial")
            </div>
            <div id="menucontainer">
                <ul id="menu">
                    <li>@Html.ActionLink("Home", "Index", "Home")</li>
                    <li>@Html.ActionLink("Identities", "Identities", "Account")</li>
                </ul>
            </div>
        </div>
        <div id="main">
            @RenderBody()
        </div>
        <div id="footer">
        </div>
    </div>
</body>
</html>

Open Views\Account\LogOn.cshtml and replace the code with the following:

@using ASPNETSimpleMVC3.Models;
@model MVC3MixedAuthenticationSample.Models.LogOnModel

@{
    ViewBag.Title = "Log On";
}

@section Styles {
    <link href="@Url.Content("~/Content/HrdPage.css")" rel="stylesheet" type="text/css" />
    <style type="text/css">
        #FormsAuth 
        {
            width:400px;
            margin:0;
            padding:0;
            float:left;
        }
    </style>
}



<h2>Log On</h2>
<p>
    Please enter your user name and password. @Html.ActionLink("Register", "Register") if you don't have an account.
</p>

@*Needed for Visual Studio Intellisense*@
@if (false)
{
    <script src="/Scripts/jquery-1.5.1-vsdoc.js" type="text/javascript"></script>
}

<script src="@Url.Content("~/Scripts/jquery.validate.min.js")" type="text/javascript"></script>
<script src="@Url.Content("~/Scripts/jquery.validate.unobtrusive.min.js")" type="text/javascript"></script>

<script type="text/javascript">
    $(function () {
        $('#AssociateCancel').click(CancelAssociation);
    });

    function CancelAssociation() {
        $.post('@Url.Action("CancelAssociation")',function(data){
           if(data) {
          
            $('#identity-provider-content').show();
            $('#AssociationMessage').hide();
           } 
           else{
            alert("Unexpected error. Try again later");
           }
        });
        
    }
    
</script>


@Html.ValidationSummary(true, "Login was unsuccessful. Please correct the errors and try again.")
@if (ViewData["HasClaim"]!=null)
{
<div id="AssociationMessage">
    <div id="AssociateMessageText">Please login to associate your account</div>
    <div id="AssociateMessageActions"><button id="AssociateCancel">Cancel</button></div> 
    <div class="clear"></div>
</div>
}
@using (Html.BeginForm()) {
    <div id="FormsAuth">
        <fieldset>
            <legend>Account Information</legend>

            <div class="editor-label">
                @Html.LabelFor(m => m.UserName)
            </div>
            <div class="editor-field">
                @Html.TextBoxFor(m => m.UserName)
                @Html.ValidationMessageFor(m => m.UserName)
            </div>

            <div class="editor-label">
                @Html.LabelFor(m => m.Password)
            </div>
            <div class="editor-field">
                @Html.PasswordFor(m => m.Password)
                @Html.ValidationMessageFor(m => m.Password)
            </div>

            <div class="editor-label">
                @Html.CheckBoxFor(m => m.RememberMe)
                @Html.LabelFor(m => m.RememberMe)
            </div>

            <p>
                <input type="submit" value="Log On" />
            </p>
        </fieldset>

    </div>
    
}
@Html.Partial("_IdentityProvidersWithServerSideCode")

Open Views\Account\_IdentityProvidersWithServerSideCode.cshtml and replace the code with the following:

@using ASPNETSimpleMVC3.Models;
@{IEnumerable<HrdIdentityProvider> Providers = null;}
@{
    if (!string.IsNullOrEmpty(ViewData["Providers"].ToString()) && (ViewData["Providers"] as IEnumerable<HrdIdentityProvider>).Count() > 0)
    { 
        Providers = ViewData["Providers"] as IEnumerable<HrdIdentityProvider>;
    }
}
  

@*Needed for Visual Studio Intellisense*@
@if (false)
{
    <script src="/Scripts/jquery-1.5.1-vsdoc.js" type="text/javascript"></script>
}

@if (Providers != null)
{
<div id="identity-provider-content"  class="@(ViewData["HasClaim"] != null ? "provider-content-hidden" : "")">
    
    
    <h2>
        Or sign in using
    </h2>
    <div id="identity-providers">
        @foreach (HrdIdentityProvider identityProvider in Providers)
        {
            <div  class="identity-provider"><button onclick="window.location='@Html.Raw(identityProvider.LoginUrl)'">@identityProvider.Name</button></div>
        }
    </div>
</div>

<div class="clear"></div>
}

Open Content/HrdPage.css and change the code to the following:


#identity-provider-content
{
    width:250px;
    text-align:center;   
    float:left;
   
}

#identity-providers
{
    text-align:center;
    width:100%;
}

div.identity-provider
{

    text-align:center;
    margin: 10px 0px 10px 0px;
}

div.identity-provider button
{
    padding:10px;
    width:150px;
    vertical-align:middle;
}

#AssociationMessage
{
    border: 1px solid #4F8A10;
    background-color: #DFF2BF;
    color:#4F8A10;
    padding:5px;
    width:400px;
}

.provider-content-hidden
{
    display:none;
}

#AssociateMessageActions
{
    float:right;
}
#AssociateMessageText
{
    float:left;
}

Open Controllers\AccountController.cs and make the following changes:

Add an action called SignIn

        /// <summary>
        /// Action used to sign user in from ACS
        /// </summary>
        /// <param name="forms"></param>
        /// <returns></returns>
        [HttpPost]
        [ValidateInput(false)]
        public ActionResult SignIn(FormCollection forms)
        {
            //Extract claims
            var principal = HttpContext.User;
            var claim = new IdentityClaim(principal.Identity as IClaimsIdentity);

            //Delete session cookie so the module cannot reset principal
            SessionAuthenticationModule sam = FederatedAuthentication.SessionAuthenticationModule;
            sam.DeleteSessionTokenCookie();

            if (claim.HasIdentity)
            {
                var db = new IdentityRepository();

                var identity = db.FindByProvderAndValue(claim.IdentityProvider, claim.IdentityValue);
                string returnUrl = GetUrlFromContext(forms) ?? Url.Action("Index", "Home", null);
                if (identity != null)
                {
                    var user = Membership.GetUser(identity.UserId);

                    if (user != null)
                    {
                        FormsAuthentication.SetAuthCookie(user.UserName, false);
                        return Redirect(returnUrl);
                    }

                }
                else
                {
                    //Save identity values for processing in the association page
                    claim.SaveToSession();

                    return RedirectToAction("LogOn", new { ReturnUrl = returnUrl });

                }

            }

            return RedirectToAction("LogOn", "Account");
        }

Still in Controllers\AccountController.cs add the following actions which will facilitate the association process.

         /// <summary>
        /// Remove pending association from session
        /// </summary>
        /// <returns></returns>
        [Authorize]
        public JsonResult CancelAssociation()
        {
            IdentityClaim.ClearSession();

            return Json(true);
        }

        /// <summary>
        /// Allows a user to remove an identity association
        /// </summary>
        /// <param name="id"></param>
        /// <returns></returns>
        [Authorize]
        public JsonResult RemoveAssociation(int id)
        {
            var db = new IdentityRepository();
            var user = Membership.GetUser();
            db.DeleteById(id,user.ProviderUserKey.ToString());

            return Json(true);
        }

        /// <summary>
        /// Allows users to manage their associated identites
        /// </summary>
        /// <returns></returns>
        [Authorize]
        public ActionResult Identities()
        {
            var db = new IdentityRepository();
            var user = Membership.GetUser();

            var identities = db.FindByUserId(user.ProviderUserKey.ToString());

            return View(identities);
        }

Add a new view to Views\Account called Identities.cshtml and insert the following code:

@model IEnumerable<MVC3MixedAuthenticationSample.UserIdentity>
@using MVC3MixedAuthenticationSample.Models

@{
    ViewBag.Title = "Identities";
}

@*Needed for Visual Studio Intellisense*@
@if (false)
{
    <script src="/Scripts/jquery-1.5.1-vsdoc.js" type="text/javascript"></script>
}

<script type="text/javascript">
    function RemoveAssociation(element, id) {
        $.post('@Url.Action("RemoveAssociation")', { id: id }, function (data) {
            if (data) {
                $(element).parent().slideUp().Remove();
            }
            else {
                alert("There was an unexpected error. Try again later");
            }
        });
    }
</script>
<h2>Identities</h2>

<div id="ProviderList">
@foreach (var item in Model) {
  <div>
    @IdentityClaim.ProviderNiceName( item.IdentityProvider )  <button onclick="RemoveAssociation(this,@item.IdentityID)">Remove</button>
  </div>
}
</div>



Replace the parameter-less LogOn() action with the following code:

        public ActionResult LogOn()
        {
            HrdClient hrdClient = new HrdClient();

            WSFederationAuthenticationModule fam = FederatedAuthentication.WSFederationAuthenticationModule;
            HrdRequest request = new HrdRequest(fam.Issuer, fam.Realm, context: Request.QueryString["ReturnUrl"]);

            IEnumerable<HrdIdentityProvider> hrdIdentityProviders = hrdClient.GetHrdResponse(request);

            var claim = new IdentityClaim();
            if (claim.HasIdentity)
            {
                ViewData["HasClaim"] = true;
            }

            ViewData["Providers"] = hrdIdentityProviders;
            return View();
        }

Replace the LogOn(LogOnModel model, string returnUrl) action with the following code:

        [HttpPost]
        public ActionResult LogOn(LogOnModel model, string returnUrl)
        {
            var claim = new IdentityClaim();

            if (ModelState.IsValid)
            {
                if (Membership.ValidateUser(model.UserName, model.Password))
                {
                   

                    if (claim.HasIdentity)
                    {
                        var db = new IdentityRepository();
                        var user = Membership.GetUser(model.UserName);
                        db.MapIdentity(user.ProviderUserKey.ToString(), claim.IdentityProvider, claim.IdentityValue);
                        IdentityClaim.ClearSession();
                    }

                    FormsAuthentication.SetAuthCookie(model.UserName, model.RememberMe);
                    if (Url.IsLocalUrl(returnUrl) && returnUrl.Length > 1 && returnUrl.StartsWith("/")
                        && !returnUrl.StartsWith("//") && !returnUrl.StartsWith("/\\"))
                    {
                        return Redirect(returnUrl);
                    }
                    else
                    {
                        return RedirectToAction("Index", "Home");
                    }
                }
                else
                {
                    ModelState.AddModelError("", "The user name or password provided is incorrect.");
                }
            }

            HrdClient hrdClient = new HrdClient();

            WSFederationAuthenticationModule fam = FederatedAuthentication.WSFederationAuthenticationModule;
            HrdRequest request = new HrdRequest(fam.Issuer, fam.Realm, context: Request.QueryString["ReturnUrl"]);

            IEnumerable<HrdIdentityProvider> hrdIdentityProviders = hrdClient.GetHrdResponse(request);

            if (claim.HasIdentity)
            {
                ViewData["HasClaim"] = true;
            }

            ViewData["Providers"] = hrdIdentityProviders;
            // If we got this far, something failed, redisplay form
            return View(model);
        }

Next open controllers\HomeController.cs and add the authorize attribute so we can test our new feature.

    [Authorize]
    public class HomeController : Controller
    {
      ...
    }

Add a new class called IdentityRepository.cs to the models folder and insert the following code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace MVC3MixedAuthenticationSample.Models
{
     
    public class IdentityRepository 
    {
        private ASPNETDBEntities _context;

        public IdentityRepository()
        {
            _context = new ASPNETDBEntities();
        }

        public int MapIdentity(string userid, string identytProvider, string identityValue)
        {
            var mapping = new UserIdentity();
            mapping.UserId = new Guid(userid);
            mapping.IdentityProvider = identytProvider;
            mapping.IdentityValue = identityValue;
            _context.AddToUserIdentities(mapping);
            _context.SaveChanges();

            return mapping.IdentityID;
        }

        public IQueryable<UserIdentity> FindByUserId(string userId)
        {
           return _context.UserIdentities.Where(i => i.UserId == new Guid(userId));
        }

        public UserIdentity FindByProvderAndValue(string IdentityProvider, string identityValue)
        {
            return _context.UserIdentities.FirstOrDefault(i => i.IdentityProvider == IdentityProvider && i.IdentityValue == identityValue);
        }

        public void DeleteById(int id, string userId)
        {
            var identity = _context.UserIdentities.FirstOrDefault(i => i.IdentityID == id && i.UserId == new Guid(userId));
            if (identity != null)
            {
                _context.DeleteObject(identity);
            }
            _context.SaveChanges();
        }

       
    }
}

At this point the solution should look like this:

Next run the project and verify that the LogOn page looks like the following:

At this point you must verify that ACS is setup properly with the project url as a valid relying party (Please see the original ACS MVC3 sample tutorial if you have not been able to set this up) and has http://%5Bprojecturl%5D/Account/SignIn as the return url. If this is not configured properly you will get a bad request exception when attempting to load the Login page.

If you have not already selected the providers you want users to authenticate against you should do so at this point. Again, there are many tutorials out there which cover this process. Facebook is the only provider which requires extra steps to be added as a provider. You can read about it here.

You should now be able to click on one of the provider buttons on the LogOn page where you will sign in using any of the providers setup in your ACS account. After signing in you will return to the following screen on your site which prompts you to login to complete the association process:

After you login you should see your new identity listed on the Identites page.

Finally, you should be able to logout then click the same provider button you clicked before to get logged in without having to enter your username and password for the site.

The sample project used in this tutorial is available on codeplex for anyone who wants it. Have fun with ACS and Asp.net memberships.

To use the sample project you will need to configure the Windows Identity framework module for your own ACS namespace.

Tutorial: Mvc application using Azure acs and forms authentication Part 1


A few months ago I created a post which described a method for adding single signon services to an asp.net mvc website using Windows Azure ACS. The workflow involved allowing users to signup using forms authentication then associating their on-site identities with their facboook, google, yahoo and windows live identities. This post is meant to be a more detailed followup which will take you through  some other things not covered in my original post.

The tutorial steps are as follows:

  • Create an asp.net mvc 3 web application with forms authentication
  • Register a new user
  • Add Azure ACS sts reference to application
  • Alter web.config to allow both sts and forms authentication
  • Add database table to store mappings between identities and user accounts
  • Create a custom class for extracting required claims
  • Create a special controller action which Azure ACS will call to sign users in
  • Create Hybrid login page with new functions
    • Login with forms
    • Login with ACS
    • Prompt user to login and complete association process

Here is the workflow from the original post:

Assumptions about audience

Please ensure you have the following before proceeding with this tutorial

  1. Windows aure account
  2. Azure Access Control Services (ACS) working
  3. Windows Identity Framework Installed and working
  4. Basic understanding of Asp.net memberships
  5. Comfortable with Asp.net MVC
  6. Visual Studio 2010
  7. You have SQL express installed with required permissions
  8. You have experience with ACS/STS either through your own projects or tutorials such as on acs.codeplex.com
Again, If you have not been able to get ACS working using other samples and tutorials then it will be very difficult for you to follow this tutorial as I will not go into detailed explanations about setting up ACS, installing the Windows Identity Framework and Asp.net memberships.


Creating the Project

We will start by creating a new Asp.net MVC 3 project. I called mine MVC3MixedAuthenticationSample

Use the “Internet Application” template

Once the project is created add a reference to the Microsoft.IdentityModel assembly.

You should have a working Asp.net MVC 3 application at this point. The asp.net database will be created in the App_Data folder when we run the application.

Now run the project and register a new user. I called mine bob and gave him the password ‘password’.

Your new user should now be logged into the site. You can also use the asp.net configuration tool to add a new user (you must build the project first). If you get any errors while attempting to add a new user you probably don’t have the correct permissions to the App_Data folder in the project or to sql express. There are several things you can try to resolve this issue that I can mention later if needed.

Our next step will be to include the newly created database in our project. This file is hidden by default so click the second icon from the left below “Solution Explorer” to show hidden files.

The database should now appear in the list of databases on the server explorer. Right click on the Tables node to add a new table.

The new table will store the mappings between the acs identities and the local users.

Remember to enable auto increment for the primary key (IdentityID)

Save table and call it UserIdentity

Now we need data access to our new table and any other tables related to it. I decided to use entity framework for this but you can use any ORM or simple ADO.NET if you like. Many people are reading this because they want to modify their existing projects so they may be using something else for data access. However, the principles are the same.

Create a new entity data model and opt to generate the entity from a database.

The “ApplicationServices” connection should already be selected. If not, select it. If it is not there, I would say add it but it not being there might be a symptom of a bigger issue.

Add the UserIdentity table to the model. I have the aspnet_users table selected but it is not required since we will be using the asp.net membership api for everything related to local users.

Now that the database and data access is setup we can add a reference to the Azure ACS STS.

Please fill in the url for the mvc application

Select “Use an existing STS” and paste in the link to your ACS metadata. This can be found in your Azure control panel.

The following claims will be used to associate users with their identities

The wizard automatically disables forms authentication so the next step is to edit our web.config and turn forms authentication back on.

Open the web.config and find the attribute passiveRedirectEnabled and set  it to false.

<wsFederation passiveRedirectEnabled="false" issuer="https://[namespace].accesscontrol.windows.net/v2/wsfederation" realm="http://localhost:52119/" requireHttps="false" />

Find the following and comment it out:

<authentication mode="None" />

Find the following and uncomment it:

<!--<authentication mode="Forms"><forms loginUrl="~/Account/LogOn" timeout="2880" /></authentication>-->

Find the following and comment it out:

<authorization>
 <deny users="?" />
 </authorization>

At this point you should be able to login to run the application and login with the user you created at the beginning.

Next we will Create a new class called IdentityClaim which we will use to extract the claims from the sts response. I added mine to the Models folder. Please note the using statement for Microsoft.IdentityModel.Claims.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using Microsoft.IdentityModel.Claims;

namespace MVC3MixedAuthenticationSample.Models
{
    public class IdentityClaim
    {
        private string _identityProvider;
        private string _identityValue;
        public const string ACSProviderClaim = "http://schemas.microsoft.com/accesscontrolservice/2010/07/claims/identityprovider";

        public IdentityClaim(IClaimsIdentity identity)
        {

            if (identity != null)
            {
                foreach (var claim in identity.Claims)
                {
                    if (claim.ClaimType == ClaimTypes.NameIdentifier)
                    {
                        _identityValue = claim.Value;
                    }
                    if (claim.ClaimType == ACSProviderClaim)
                    {
                        _identityProvider = claim.Value;
                    }

                }
            }

        }

        public IdentityClaim()
        {
            _identityProvider = HttpContext.Current.Session["IdentityProvider"] as string;
            _identityValue = HttpContext.Current.Session["IdentityValue"] as string;
        }

        public bool HasIdentity
        {
            get
            {
                return (!string.IsNullOrEmpty(_identityProvider) && (!string.IsNullOrEmpty(_identityValue)));
            }
        }

        public string IdentityProvider
        {
            get
            {
                return _identityProvider;
            }
        }

        public string IdentityValue
        {
            get
            {
                return _identityValue;
            }
        }

        internal void SaveToSession()
        {
            HttpContext.Current.Session["IdentityProvider"] = _identityProvider;
            HttpContext.Current.Session["IdentityValue"] = _identityValue;
        }

        public static void ClearSession()
        {
            HttpContext.Current.Session.Remove("IdentityProvider");
            HttpContext.Current.Session.Remove("IdentityValue");
        }

        public static string ProviderNiceName(string identityProivder)
        {
            if (identityProivder.ToLower().Contains("windowslive"))
                return "Windows Live";

            if (identityProivder.ToLower().Contains("facebook"))
                return "Facebook";

            if (identityProivder.ToLower().Contains("yahoo"))
                return "Yahoo";

            if (identityProivder.ToLower().Contains("google"))
                return "Google";

            return identityProivder;
        }
    }

}

In my next post I will talk about creating the new controller action that will support Azure ACS signin and modifying the generated logOn page to include both the ACS buttons and the regular forms authentication option.

A very simple Windows Azure AppFabric Cache Class


I have an application on windows azure which has both a worker and a webrole. I needed a way to send basic throw away info to the webrole from the worker role, so I decided to use windows azure cache. I created this very simple class which allows you to initialize a connection to the cache and use it for basic adding and reading of data.

Before using this class you need to AppFabric SDK from here. After installing the SDK Add a reference to the caching api by browsing and selecting the following file:

[SDK Install Path]\V1.5\Assemblies\NET4.0\Cache\Microsoft.ApplicationServer.Caching.Client.dll

Setup your azure cache instance then insert your access key and azure cache url in the code below.

Here is the code:


using System;
using System.Collections.Generic;
using Microsoft.ApplicationServer.Caching;
using System.Security;

namespace Yoursite.Azure.Wrappers
{
    public static class CacheStore
    {
        private static DataCache _cache;

        static CacheStore()
        {
            // Declare array for cache host.
            DataCacheServerEndpoint[] servers = new DataCacheServerEndpoint[1];
            servers[0] = new DataCacheServerEndpoint("[YOUR-NAME-SPACE].cache.windows.net", 22243);

            // Setup DataCacheSecurity configuration.
            string strACSKey = "[INSERT KEY HERE]";
            var secureACSKey = new SecureString();
           
            foreach (char a in strACSKey)
            {
                secureACSKey.AppendChar(a);
            }
            secureACSKey.MakeReadOnly();
            DataCacheSecurity factorySecurity = new DataCacheSecurity(secureACSKey,true);

            // Setup the DataCacheFactory configuration.
            DataCacheFactoryConfiguration factoryConfig = new DataCacheFactoryConfiguration();
            factoryConfig.Servers = servers;
            factoryConfig.SecurityProperties = factorySecurity;
           

            // Create a configured DataCacheFactory object.
            DataCacheFactory cacheFactory = new DataCacheFactory(factoryConfig);

            // Get a cache client for the default cache.
            _cache = cacheFactory.GetDefaultCache();

        }

        public static void SetValue(string key,object value)
        {
            _cache[key] = value;
        }

        public static T GetValue<T>(string key)
        {
            return (T)_cache[key];
        }
    }
}

Usage:

var saveme = 856;
//Store value
CacheStore.SaveValue("savemekey",saveme);

//Retrieve value
var readme = CacheStore.GetValue<int>("savemekey");

Enjoy! Don’t forget to catch exceptions. My error logs show exceptions while attempting to communicate with the azure caching server every now and then.