Saving time via Logic Apps: a real world example

Introduction

At Codit, I manage the blog. We have some very passionate people on board who like to invest their time to get to the bottom of things and – also very important – share it with the world!
That small part of my job means I get to review blog posts before publishing on a technical level. It’s always good to have one extra pair of eyes reading the post before publishing it to the public, so this definitely pays off!

An even smaller part of publishing blog posts is making sure they get enough coverage. Sharing them on Twitter, LinkedIn or even Facebook is part of the job for our devoted marketing department! And analytics around these shares on social media definitely come in handy! For that specific reason we use Bitly to shorten our URLs.
Every time a blog post gets published, someone needed to add them manually to out Bitly account and send out an e-mail. This takes a small amount of time, but as you can imagine it accumulates quickly with the amount of posts we generate lately!

Logic Apps to the rescue!

I was looking for an excuse to start playing with Logic Apps and they recently added Bitly as one of their Preview connectors, so I started digging!

First, let’s try and list the requirements of our Logic App to-be:

Must-haves:

  • The Logic App should trigger automatically whenever a new blog post is published.
  • It should create a short link, specifically for usage on Twitter.
  • It also should create a short link, specifically for LinkedIn usage.
  • It should send out an e-mail with the short links.
  • I want the short URLs to appear in the Bitly dashboard, so we can track click-through-rate (CTR).
  • I want to spend a minimum of Azure consumption.

Nice-to-haves:

  • I want the Logic App to trigger immediately after publishing the blog post.
  • I want the e-mail to be sent out to me, the marketing department and the author of the post for (possibly) immediate usage on social media.
  • If I resubmit a logic app, I don’t want new URLs (idempotency), I want to keep the ones already in the Bitly dashboard.
  • I want the e-mail to appear as if it was coming directly from me.

Logic App Trigger

I could easily fill in one of the first requirements, since the Logic App RSS connector provides me a very easy way to trigger a logic app based on a RSS feed. Our Codit blog RSS feed seemed to do the trick perfectly!

Now it’s all about timing the polling interval: if we poll every minute we get the e-mail faster, but will spend more on Azure consumption since the Logic App gets triggered more… I decided 30 minutes would probably be good enough.

2017-06-09 00_30_33-Logic Apps Designer - Microsoft Azure

Now I needed to try and get the URL for any new posts that were published. Luckily, the links – Item provides me the perfect way of doing that. The Logic Apps designer conveniently detects this might be an array of links (in case two posts get published at once) and places this within a “For each” shape!

2017-06-09 00_33_34-Logic Apps Designer - Microsoft Azure

Now that I had the URL(s), all I needed to do was save the Logic App and wait until a blog post was published to test the Logic App. In the Logic App “Runs history” I was able to click through and see for myself that I got the links array nicely:

2017-06-09 00_40_08-Logic app run - Microsoft Azure

Seems there is only one item in the array for each blog post, which is perfect for our use-case!

Shortening the URL

For this part of the exercise I needed several things:

  • I actually need two URLs: one for Twitter and one for LinkedIn, so I need to call the Bitly connector twice!
  • Each link gets a little extra information in the query string called UTM codes. If you are unfamiliar with those, read up on UTM codes here. (In short: it adds extra visibility and tracking in Google Analytics).
    So I needed to concatenate the original URL with some static UTM string + one part which needed to be dynamic: the UTM campaign.

For that last part (the campaign): we already have our CMS cleaning up the title of a blog post in the last part of the URL being published! This seems ideal for us here.

However, due to lack of knowledge in Logic Apps-syntax I got so frustrated and created an Azure Function to do just that (extract the interesting part from the URL):

2017-06-09 00_49_30-Logic app run - Microsoft Azure

I wasn’t pleased with this, but at least I was able to get things running…
It however meant I needed extra, unwanted, Azure resources:

  • Extra Azure storage account (to store the function in)
  • Azure App Service Plan to host the function in
  • An Azure function to do the trivial task of some string manipulation.

After some additional (but determined) trial and error late in the evening, I ended up doing the same in a Logic App Compose shape! Happy days!

2017-06-09 00_52_35-Logic Apps Designer - Microsoft Azure

It takes the URL, splits it into an array, based on the slash (‘/’) and takes the part which is interesting for my use-case. See for yourself:

2017-06-09 00_55_30-Logic app run - Microsoft Azure

Now I still needed to concatenate all pieces of string together. The concat() function seems to be able to do the trick, but an even easier solution is to just use another Compose shape:

2017-06-09 00_56_35-Logic Apps Designer - Microsoft Azure - Twitter2017-06-09 00_56_49-Logic Apps Designer - Microsoft Azure - LinkedIn

Concatenation comes naturally to the Compose shape!

Then I still needed to create the short links by calling the Bitly connector:

2017-06-09 00_59_33-Logic Apps Designer - Microsoft Azure - Bitly

Let’s send out an e-mail

Sending out e-mail, using my Office365 account is actually the easiest thing ever:

2017-06-09 01_00_45-Logic Apps Designer - Microsoft Azure - Mail

Conclusion

My first practical Logic App seems to be a hit! And probably saves us about half an hour of work every week. A few hours of Logic App “R&D” will definitely pay off in the long run!

Here’s the overview of my complete Logic App:

2017-06-09 01_01_41-Logic Apps Designer - Microsoft Azure - Complete Logic App.png

Some remarks

During development however, I came across – what appear to me – some limitations :

  • The author of the blog post is not in the output of the RSS connector, which is a pity! This would have allowed me to use his/her e-mail address directly or, if it was his/her name, to look-up the e-mail address using the Office 365 users connector!
  • I’m missing some kind of expression shape in Logic Apps!
    Coming from BizTalk Server where expression shapes containing a limited form of C# code are very handy in a BizTalk orchestration, this is something that should be included one way or the other (without the Azure function implementation).
    A few lines of code in there is awesome for dirty work like string manipulation for example.
  • It took me a while to get my head around Logic Apps syntax.
    It’s not really explained in the documentation when or when not to use @function() or @{function()}. It’s not that hard at all once you get the hang of it. Unfortunately it took me a lot of save errors and even some run-time errors (not covered at design time) to get to that point. Might be just me however…
  • I cannot rename API connections in my Azure Resource Group. Some generic names like ‘rss’, ‘bitly’ and ‘office-365’ are used. I can set some properties so they appear nicely in the Logic App however.
  • We have Office365 Multi-Factor Authentication enabled at our company. I can authorize the Office365 API connection, but this will only last for 30 days. I might need to change to an account without multi-factor authentication if I don’t want to re-authorize every 30 days…

Let me know what you think in the comments! Is this the way to go?
Any alternative versions I could use? Any feedback is more than welcome.

In a next blog post I will take some of our Logic Apps best practices to heart and optimize the Logic App.

Have a nice day!
Pieter

 

 

Fixing DistributedCOM launch issues

Recently, when checking some event logs on a clustered SQL environment, I encountered the following error in the System event log:

The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID
{806835AE-FD04-4870-A1E8-D65535358293}
and APPID
{EE4171E6-C37E-4D04-AF4C-8617BC7D4914}
to the user DOMAIN\user SID (<SID>) from address LocalHost (Using LRPC) running in the application container Unavailable SID (Unavailable). This security permission can be modified using the Component Services administrative tool.

2017-05-18 17_58_02-mRemoteNG - mgRemoteNG.xml - SQL PRD 1

This happened twice … at the beginning of every minute.

Unaware what the Application ID listed was, I was able to retrieve the list of DCOM applications with the following PowerShell script:

$strComputer = “.”

$colItems = get-wmiobject -class “Win32_DCOMApplication” -namespace “root\CIMV2” -computername $strComputer

foreach ($objItem in $colItems) {
write-host “Application ID: ” $objItem.AppID
write-host “Caption: ” $objItem.Caption
write-host “Description: ” $objItem.Description
write-host “Installation Date: ” $objItem.InstallDate
write-host “Name: ” $objItem.Name
write-host “Status: ” $objItem.Status
write-host
}

With this, it was easy to track the application ID listed back to Microsoft SQL Server Integration Services. The user mentioned in the error was also a user only used for the SQL Server Agent jobs.

This led me to a SQL Server Job which ran every minute which uses SSIS in some of it’s steps. Strangely, the SQL Server job did not fail and kept working as expected.

After some Googling, I managed to fix the issue as so:

  • Start “Component Services”
  • Choose Computers
  • Choose “My Computer”
  • Choose “DCOM Config”
  • Choose the service matching the APPID. In this case Microsoft SQL Server Integration Services 12.0
    2017-05-18 18_05_32-mRemoteNG - mgRemoteNG.xml - SQL PRD 1
  • Click the Security tab
  • Go to Launch and Activation Permissions, Customize, Edit and add the account mentioned. Check the check boxes as below:
    2017-05-18 18_07_13-mRemoteNG - mgRemoteNG.xml - SQL PRD 1
  • Don’t forget to restart the application service (here: Microsoft SQL Server Integration Services 12.0) in order to be sure that the service applied the changes.

Once done, the errors stopped occurring and the event log was kept nice and tidy.

Hope this helps at least someone.

Cheers,
Pieter

Troubleshooting VSTS Build Agent configuration

While figuring out the BizTalk 2016 Feature Pack 1 ALM feature, the setup & configuration of the Visual Studio Team Services (VSTS) build agent gave me some issues while configuring:

2017-05-01 13_29_31-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

The error reads:

Exception of type ‘Microsoft.VisualStudio.Services.OAuth.VssOAuthTokenRequestException’ was thrown.

After searching for quite some time, I noticed the clock on my Virtual Machine was off. Strange, since Hyper-V will synchronized it quite fast after a resumed save.

Funny enough, after retrying, this seemed to do the trick. Long story short: if you have the above error, make sure the clock of your machine is synchronized.

Cheers,
Pieter

BizTalk 2016 Feature Pack 1 – Installation walk-through & notes

Some side-notes on the BizTalk Server Feature Pack 1:

  • BizTalk Server 2016 Feature Pack 1 does not require you to have BizTalk Server 2016 CU1 installed.
  • Cumulative updates will take into account any BizTalk Server Feature Packs installed and will not break them. This effectively means that you do not need to re-install a CU after a Feature Pack installation.
    This does leave me wonder what will happen once they fix a bug in a Feature Pack in a CU.
  • You have to install it on each BizTalk Server in a BizTalk Group in order to upgrade the entire group. Preferably, you would keep all host instances disabled until all servers are updated. Keep a full backup of your BizTalk databases set nearby, just in case.

Below you can find a walk-through for the BizTalk Server 2016 Feature Pack 1 installation. This might serve someone for a document on how to install the feature pack.

Step 1: Download BizTalk 2016 Feature Pack 1

The BizTalk Server 2016 Feature Pack 1 can be downloaded from this URL:
https://www.microsoft.com/en-us/download/details.aspx?id=55100.

On the Microsoft download page it is actually named “BizTalk Server 2016 Update 1” and I don’t think that was a very good choice to be honest. I believe it will create confusion between BizTalk Server cumulative update packages and feature packs. Nevertheless, that’s the one you want!

Step 2: Install BizTalk 2016 Feature Pack 1

Start the executable:

2017-04-27 02_08_31-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Click Next:

2017-04-27 02_09_11-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Accept the EULA and click Next:

2017-04-27 02_09_36-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

If possible, participate in the CEI program:

2017-04-27 02_10_00-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Not sure what they tried to achieve with this, but just click Next:

2017-04-27 02_10_11-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Click Yes to agree with the restart of your Host Instances, the ESSO service, IIS Admin Service (if installed), WMI, etc…

2017-04-27 02_10_22-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Patiently wait out until the install finishes:

2017-04-27 02_10_43-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

2017-04-27 02_10_56-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Click Finish to complete the installation.

2017-04-27 02_12_50-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Step 3: Verifying the installation

Verify you can find the following event log in the event viewer:

Product: Microsoft BizTalk Server 2016 – Update ‘Microsoft BizTalk Server 2016 Hotfix [KB 4014788]’ installed successfully.

2017-04-27 02_14_56-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Windows Installer installed an update. Product Name: Microsoft BizTalk Server 2016. Product Version: 3.12.774.0. Product Language: 1033. Manufacturer: Microsoft Corporation. Update Name: Microsoft BizTalk Server 2016 Hotfix [KB 4014788]. Installation success or error status: 0.

2017-04-27 02_15_24-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Windows Installer reconfigured the product. Product Name: Microsoft BizTalk Server 2016. Product Version: 3.12.774.0. Product Language: 1033. Manufacturer: Microsoft Corporation. Reconfiguration success or error status: 0.

2017-04-27 02_15_52-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

When you check Programs & Features, you can easily see if the Feature Pack is installed or not:

2017-04-27 02_40_14-mRemoteNG - mgRemoteNG.xml - BizTalk 2016 - Demo

Cheers,
Pieter Vandenheede

SQL Server Agent 2016 does not start with Shared Memory disabled

If you need to install BizTalk Server 2016, there is a big chance you will also install SQL Server 2016. Not only does SQL Server 2016 support the AlwaysOn capabilities that BizTalk Server 2016 might need, it comes with a much-welcomed upgrade to the support life-cycle. At the moment of writing, the latest service pack of SQL Server 2016 is SP1, which offers support until Jul 13 2021.

Installing BizTalk Server properly will require you to make some changes to the SQL Server Configuration as well. Specifically, you might want/need to disable the Shared Memory Protocol.

Shared Memory Protocol

The SQL Server Shared Memory Protocol per the MSDN documentation around SQL protocols:

Shared memory is the simplest protocol to use and has no configurable settings. Because clients using the shared memory protocol can only connect to a SQL Server instance running on the same computer, it is not useful for most database activity. Use the shared memory protocol for troubleshooting when you suspect the other protocols are configured incorrectly.

Basically, in the case your application is not hosted on the same server as your SQL Database Engine, it makes zero sense to enable the protocol. Therefore, it is considered good practice to disable the protocol and only enable the Named Pipes and TCP/IP protocol:

2017-03-29 14_53_26-mRemoteNG - mgRemoteNG.xml - SQL PROD

The above screenshot gives an overview of how the protocol configuration at server side should look like. At Codit, we disable the protocol at server side, since it might pose somewhat of a security risk.

SQL Server Agent not running

While the above has been a good practice for many years and many different BizTalk versions and never gave us any issues, for SQL Server 2016 this meant that the SQL Server Agent (windows service) was stopping unexpectedly. Something that came to my attention while trying to configure the BizTalk SQL Jobs.

The below is a screenshot of the event log and the error logged:

2017-04-12 16_30_41-Clipboard

The error:

SQLServerAgent could not be started (reason: Unable to connect to server ‘SOMESERVER’; SQLServerAgent cannot start).

(Re)starting the SQL Agent service works fine, but after about a minute, we get the same issue and the service crashes.

After some google-Fu, I finally managed to find the culprit due to a message in the SQL Server log:

2017-03-27 13:33:18 – ! [165] ODBC Error: 0, Login timeout expired [SQLSTATE HYT00]

And with some further research, it seemed the ODBC version that ships with SQL Server 2016, which is ODBC v13.0, contains a bug that will prevent the SQL Server Agent from running with Shared Memory disabled!

ODBC Drivers

To check the version of the ODBC drivers on your machine, there is actually a page on Microsoft Docs: https://docs.microsoft.com/en-us/sql/database-engine/configure-windows/check-the-odbc-sql-server-driver-version-windows

Mine said:

2017-04-12 16_47_22-Clipboard

Note: 2015.130.x = v13.0

We now need to upgrade ODBC to version v13.1. You can download ODBC v13.1 from the Microsoft website: https://www.microsoft.com/en-us/download/details.aspx?id=53339

After upgrading, the Drivers tab mentions v2015.131.x = v13.1

2017-04-13 09_09_25-Clipboard

And, as expected (and hoped), the SQL Server Agent service now start successfully and keeps running:

2017-04-13 09_10_18-Clipboard

Let me know if this helped you or if you have any questions.

Cheers!
Pieter

Earned MCSA SQL Server 2012/2014!

msft_partner2016Codit is a Microsoft Gold Partner in Application Development, Application Integration and Cloud Platform and a Silver Partner for Data Analytics.

This means we are required to have staff certified, which in turn requires us to take (and pass) Microsoft exams once in a while. This is not mandatory on a personal level, but still it is necessary for some people in order to retain or even improve our Microsoft Partnership.

Although it may be a lot of work to prepare for the exam, most of us are quite keen on getting certified actually. As one of the more ‘senior’ guys at Codit, I try to get certified or pass an exam at least once a year. Although it means that you will need to do some research, buy and read … and re-read the book(s), spend additional time looking up facts on MSDN, etc… I do agree it’s a nice recognition to have passed an exam.

mcsa_sql

In the last years I have been able to pass 70-461: Querying Microsoft SQL Server 2012/2014 and 70-462 Administering Microsoft SQL Server 2012/2014 Databases. Both exam preparations were actually great ways to get some recognition on working with SQL Server since 2001 (the year I graduated).
Along the way I have learned and picked up a lot from (ex-)colleagues, numerous blogs and various sources and books. Still, I was shocked at how much I was still able to learn by grabbing and reading the preparation books, doing the exercises, watching SQL courses on Pluralsight and looking up more information. It was an invaluable experience, learning so much. Some of it already came in very handy during the last year.

Since a couple of years Microsoft features the MCSA. The Microsoft Certified Solutions Associate. One of the tracks is MCSA: SQL Server 2012/2014.
It requires you to pass 3 exams: the ones mentioned above (70-461 and 70-462) and one other. Although it mentions 70-463: Implementing a Data Warehouse with Microsoft SQL Server 2012/2014 as the third exam, you can actually substitute it with any of the following as well:

Since I don’t feel that much for BI in SQL Server, I thought it was a good idea to certify for C# and took the Programming in C# exam last week.

mcsa-sql

With great pride I can now tell you that I also passed this last exam and earned the MCSA: SQL Server 2012/2014 title!

It even comes with a badge! :^)

I can only recommend it to be honest: I learned a great deal and find it very valuable in my day-to-day work.

If you have any questions, let me know and I’ll be happy to share my experience with you.

Now that SQL Server 2016 is out, Microsoft is already working on an upgrade path. Not one, but two! I know what is on the menu for 2017-2018! ;^)

BTUG.be XL – BizTalk 2016 Slidedeck

Earlier this evening, I had a great time at BTUG.be, while presenting my session on BizTalk 2016.

I presented the new features in BizTalk Server 2016 RTM and a few takeaway from SQL Server 2016. More specifically, and in-depth, on SQL Server AlwaysOn support for BizTalk Server 2016 on-premise and in the Azure cloud, as well as an intro on the new Logic App adapter and how to install and connect it to your on-premise BizTalk Server.

Thanks to my company, Codit.eu, for providing me the opportunity to be there!

As promised there, please find my slide deck below via SlideShare:

Contact me if you have any questions regarding the slides, I’d be happy to answer you.

The other speakers there were Glenn Colpaert (session about Azure Functions), Kristof Rennen (session on Building scalable and resilient solutions using messaging) and Nino Crudele (session on Holistic approaches to Integration).

As always, it was nice to talk to the people present. A big thank you to BTUG.be for having me again!

Enjoy the slide deck!

Pieter