Hands On: Application Integration for Dynamics CRM using Azure Service Bus
Part 1: Posting Messages from CRM to the Azure Service Bus
Over the years I have done a number of integrations between Microsoft Dynamics CRM and various other systems. For the most part those integrations have been accomplished through middleware like Dynamics Connector, Scribe, SSIS, and even roll-your-own code based solutions. It hasn’t been until recently that I have had the chance to work with a client that wanted to use Microsoft Azure Service Bus as their ESB of choice. When I heard this I thought “Great, finally an integration mechanism that CRM is designed to work with, this should be a piece of cake”. Even though I didn’t have any experience working with CRM and the Azure Service Bus, I knew there was a walkthrough for doing this in the Dynamics CRM SDK, so I figured “How hard could it be?”. Never content to learn by reading about a subject, I couldn’t wait to get my hands on the parts and start coding. Turns out the SDK walkthrough doesn’t really explain much of anything, and either glosses over or completely omits important pieces of this puzzle. I am hoping what you find here will be more informative.
The idea here is to create an integration test where records created in a local (on premise) CRM development organization will be posted out to Azure on a specific Topic. From there, the information can be picked up and integrated into a client application that is built to ‘subscribe’ to this particular topic. Building the integration on an On Premise CRM installation will allow us to control all aspects of the installation, as well as to resolve some of the trickier aspects of ACS based security that are handled in part automatically in an Online CRM instance. You can use the SAS model for integration security if you use custom Plug-Ins in CRM for integration. The goal here is to set up the integration ‘Out of the Box’ in the CRM side, so we will not go into custom Plug-Ins here. So we will use the native Azure Service Bus connection provided as part of the Dynamics CRM install. Posting to a Topic in the Azure Service Bus also represents a more likely real-world scenario here, particularly where the intent of the client is to have an ESB based on Publish and Subscribe principals.
What You Will Need
To follow along you will need the following:
- Microsoft Dynamics CRM 2016 On Premise (or Online) – Free Trial
- Microsoft Azure Subscription – Free Trial
- Microsoft Visual Studio 2013 (or later) – Community Edition
- Windows PowerShell including the Microsoft PowerShell Snap-in for Microsoft Dynamics CRM (installed with Dynamics CRM) – Windows Management Framework 4.0
- PowerShell Azure Command Line Utilities – Installation
- Microsoft Dynamics CRM Plug-In Registration Utility (Available through the Microsoft Dynamics CRM SDK)
What you should not need for this little experiment is to shell out any of your own hard-earned cash. Thus, All of these tools and applications are available at no cost, at least on a trial basis. See the links I have provided.
Dynamics CRM ACS Security
The default Azure Service Bus connection included in CRM implements the ACS (Access Control Service) security protocol. This protocol requires a valid Certificate installed on the CRM host, registered in CRM and in the Azure Service Bus namespace. If you are using an Online version of CRM for this, the Certificate is provided for you. Go to the Settings >> Customizations >> Developer Resources. You will be provided with a link to download the Certificate:
Great, you can download the Certificate from here and you are ready to roll. You can skip to the Azure Service Bus Configuration section here. But wait. We said we were going to target an On Premise installation of CRM. If you go to the same location in an On Premise CRM UI you will see no section entitled Windows Azure Service Bus Issuer Certificate. That is because for an On Premise installation of CRM you must install your own certificate first. Now, the CRM SDK glosses over this by saying simply: “For Microsoft Dynamics CRM 2016 on-premises and IFD installations, you can purchase a private certificate from an issuing authority.”. The key word here being “purchase”. In most cases these do not come cheaply. And if you recall the emphasis for this hands-on is that you should not have to spend any money. If you have one of these certificates lying around, you can skip the next section. Otherwise, I have included here the following:
Generating a Self-Signed Certificate for Testing
Fortunately, I found this blog article by Adam Conkle. He provides some code you can save as a .ps1 (PowerShell) script. When you run it, make sure you do so as an Administrator. It suppresses error messages so you won’t immediately be aware of any problems if they exist. Change “SilentlyContinue” to “Stop” in the code if you want to see errors. This as well as the remaining steps in this process will need to be run as an administrator directly on the Microsoft Dynamics CRM server. Running this should look like the following:
The result in this case is that a private Certificate named CrmAzureSbTest has been added to your Computer’s Cert Store. Find it by opening the Microsoft Management Console (MMC) and adding the Certificates snap-in. It should appear as so:
Right click on this entry, then select All Tasks >> Manage Private Keys… You will see the following:
Make sure that the account that is running the Microsoft Dynamics CRM Asynchronous Processing Service (in the example: NETWORK SERVICE) has been given read permissions to this Certificate here. If you are not sure what account the Microsoft Dynamics CRM Asynchronous Processing Service is running under check the ‘Log On As’ account under MMC Services… on the CRM server.
Finally, export this Certificate to a file we can use to configure CRM for Azure Service Bus integration. Once again, right click on the Certificate we generated in the Certificates snap-in. Select All Tasks >> Export… This will open a Certificate Export Wizard. Select ‘Do not export private key’ and ‘Base 64 encoded x.509 (.CER)’ Then choose an appropriate file name and location. Congratulations, you now have a free Certificate file you can use for testing purposes. You should now be ready to configure CRM to use it.
Configuring On Premise CRM Azure Service Bus to use your Certificate
If you have a certificate file from an issuing authority you can install it on the CRM Server by double-clicking the file and clicking on the Install Certificate… button. Select a location on the local machine, and make sure the CRM Async Service account has read access (see above). Otherwise you should have the certificate file we generated in the previous step.
For this step we will be using PowerShell and invoking some commands from the CRM Management Snap-in. Open PowerShell as Administrator, and issue the following command: “Add-PSSnapin Microsoft.Crm.Powershell”. This will load up the PowerShell Crm Management Shap-in. From here issue the following command to load the certificate for use in your integration: “Set-CrmCertificate –CertificateType AppFabricIssuer –Name “Root Agency” –StoreName My –StoreLocation LocalMachine –StoreFindType FindBySubjectDistinguishedName –DataFile <Your Cert File Location>”. Assuming this command was successful, you should be able to get the certificate information now stored in the CRM database by issuing the following command: “Get-CrmCertificate”. The result should be something like the following:
Verify that the certificate is now being used by your On Premise CRM Org by opening the CRM UI and navigating to Settings >> Customizations >> Developer Resources. You should now have a section entitled “Connect this instance of Dynamics CRM to Azure Service Bus”:
Make note of the Issuer Name, as we will use this to configure Azure.
Azure Service Bus Configuration
If you have an Azure Services Account you can go the the Management Portal for that account. If you don’t already have one you can use the link I gave above to create a trial account to use here. The logical next step here would be to go to Azure Service Bus area, and use the UI to create a new Namespace to use for our integration. If you did that however, you would notice that Namespaces created this way no longer support ACS connections natively. Doh! Not logical. So what now? Looks like we’re back to using PowerShell to create an Azure Service Bus Namespace that will support ACS like CRM expects. Make sure you have installed the Azure PowerShell Command Utilities from the above link. Open your PowerShell command interface as Administrator. Now log in to Azure using the following command: “azure login”. You will be asked to open the url: https://aka.ms/devicelogin and you will be given a code to enter there to verify access from your machine. You will then be provided with a UI for login using your azure account. Assuming this is successful (you get a green OK in the command line), you can continue creating a new Azure Service Bus Namespace that will invoke ACS by entering the following command: “New-AzureSBNamespace –Name CrmAzureSbTest –Location “North Central US” –NamespaceType Messaging –CreateACSNamespace $true”. If you wish more detail on the available parameters (including available regions) enter the command New-AzureSBNamespace without parameters and enter !? for help. The result should look like this:
Now you should be able to go out to the Azure Management Portal and check that the namespace we created. Double-click on the namespace we just created and click on the Topics tab across the top to create a new Topic for CRM to post to. Click on the CREATE A NEW TOPIC link. Click on the QUICK CREATE link:
Enter your Topic Name, e.g.: “CrmAzureSbTestTopic”. Then click the CREATE A NEW TOPIC link. The pop-up window (above) should close with a message that the topic was successfully created. The new topic should now be listed on the topics tab with a check mark showing ‘Active’. Although our new Azure Service Bus Namespace is not yet completely configured for integration with Microsoft Dynamics CRM we are finished with PowerShell and Azure Management Portal configuration. If you have read the CRM SDK documentation you may note that there are a number of additional steps included about structuring the ACS configuration through the Azure Management Portal to work with CRM. I have tried this and it doesn’t actually seem to work. Better to skip all this nonsense and complete the configuration process using the Microsoft Dynamics CRM Plug-in Registration Tool as described below.
Adding an Azure Service Bus Connection Through the Plug-in Registration Tool:
If you have not yet downloaded the Microsoft Dynamics CRM 2016 SDK do so now using the above link. The Plug-in Registration Tool is located in: SDK >> Tools >> PluginRegistration >> PluginRegistration.exe of the downloaded contents. Open the tool and click on CREATE A NEW CONNECTION to connect to your desired On Prem or Online Organization using your administrative account credentials. When you have successfully connected, at the top of your Org tab you will see a Register drop-down link. Select ‘Register New Service Endpoint’ here. You will be presented with the following configuration form:
You can name the Service Endpoint connection whatever you wish, as well as provide any desired description. Note the following:
- Solution Namespace: This will be the root namespace we created above using the Azure PowerShell command.
- Path: This will be the name of the Topic we created above through the Azure Management Portal.
- Contract: This is the type of Azure Service Bus connection we are creating. In our case we are connecting and posting messages to a Topic.
- Claim: If you want to send information on the posting User Id and / or additional posting User Information along as part of the message posting you can select one of those options from this drop-down.
Click ‘Save & Configure ACS’. You will be presented with the following configuration window:
Enter the following values:
- Management Key: Go back to the Azure Management Portal. Select the Azure Service Bus Namespace we created from the list of available Namespaces there. At the bottom of the default namespace screen there is a link for CONNECTION INFORMATION. Clicking on that will give you a screen including ACS Connection String, Default Issuer, and Default Key. Copy out the Default Key and paste it in here.
- Certificate File: Use the ellipses to find the location of the file issued by the certificate authority or that you exported from the Certificate mmc above.
- Issuer Name: Issuer Name from the CRM Developer Resources Page.
Click on the ‘Configure ACS’ button. The center content of the form should populate with configuration information. If you have done everything right you should not get any error messages here. You can close this configuration window. You will then be back at the Service Endpoint Registration window (above). Here you can click on the ‘Save & Verify Authentication’ button. You should see a window with the title (after validation processing) “Verifying Authentication: Success”. Congratulations, you have now successfully connected Microsoft Dynamics CRM to the Azure Service Bus Namespace and Topic you have created.
Register Plug-in Steps to Post to the Azure Service Bus Topic
Whew, almost there. At this point all we have to do is to pick the CRM Entities and Message Content we want to send to the Azure Service Bus Namespace Topic. The native CRM Azure–aware Plug-in that does this transfers the information in the form of a serializable RemoteExecutionContext object. Registering this Plug-in is the same as registering any Plug-in for CMS. We will use the Plug-in Registration Tool. Using this tool, Right-click on the Service Endpoint we created above, then select ‘Register New Step’ option to select the desired Message and Entity. In our case we will select the ‘Create’ message for the ‘contact’ entity:
Note we are opting to do this Post-operation, and asynchronously. Therefore, whenever we create a new contact record in CRM this plugin should connect to the Azure Service Bus Namespace we created and post a message to the Topic crmazuresbtesttopic. The message will be of type Microsoft.ServiceBus.Messaging.BrokeredMessage. It will have as its body content the Microsoft.Xrm.Sdk.RemoteExecutionContext related to the creation of the new contact record. The Azure-aware plugin is entity and message agnostic, which means on the plus-side we can create it for nearly every entity and message type in CRM. On the other hand it means any service that subscribes to our posts will have to know something about the contents we are posting in order to sort it all out. More about that in part 2 of this post. For now all we need to do is make sure that the newly registered Plug-in works to post to Azure without error.
Test the Posting
The test is simple. Go out to CRM and (in this case) create a new contact record. After creating a new contact you should have an entry in the CRM System Jobs: Settings >> System Jobs, indicating that the asynchronous plugin posting the create to Azure has been executed. Hopefully it is there with a status reason of ‘Succeeded’. If it has failed, you can open it and check the details for any errors in submission. In Azure, you can check the dashboard for the Topic that we created. You should (eventually) see the message or messages that you created reflected in the performance monitor graph:
Coming in Part 2
Obviously this blog covers only half the equation. The other half is to write a listener that will subscribe to the Azure Service Bus Topic we created, then parse out the BrokeredMessage and do something with the data. While the CRM SDK provides some code that will do that, the actual implementation is a mystery. In part 2 we will cover a practical scenario of creating an Azure based service that will listen for messages and parse out the content.
Want to chat with someone on the Hitachi Solutions team? Click Here to get the conversation started!
On-Premises vs. Cloud CRM
A guide to help you determine the best solutions for you.