wiki

Codit Wiki

Loading information... Please wait.

Codit Blog

Posted on Thursday, May 4, 2017 1:40 PM

Toon Vanhoutte by Toon Vanhoutte

Recently, the product team released a first feature pack for BizTalk Server 2016. Via this way, Microsoft aims to provide more agility into the release model of BizTalk Server. The feature pack contains a lot of interesting features, of which the brand new Management & Operational API is an important one. Let's have a look at what's included!

The documentation of the Management API can be found here.  In short: almost everything you can access in the BizTalk Administration Console is now available in the BizTalk Management API.  The API is very well documented with Swagger, so it's pretty much self-explaining.  

What is included?

A complete list of available operations can be found here.

Deployment

There are new opportunities on the deployment side. Here are some ideas that popped into my mind:

  • Dynamically create ports. Some messaging solutions are very generic. Adding new parties is sometimes just a matter of creating a new set of receive and send ports. This can now be done through this Management API, so you don't need to do the plumbing yourself anymore.

  • Update tracking settings. We all know it quite difficult to keep your tracking settings consistent through all applications and binding files. The REST API can now be leveraged to change the tracking settings on the fly to their desired state.

Runtime

Also the runtime processing might benefit from this new functionality. Some scenarios:

  • Start and stop processes on demand. In situations that the business wants to take control on when certain processes should be active, you can start/stop receive/send ports on demand. Just a small UI on top of the Management API, including the appropriate security measures, and you're good to go!

  • Maintenance windows. BizTalk is in the middle of your application landscape. Deployments on backend applications, can have a serious impact on running integrations. That's why stopping certain ports during maintenance windows is a good approach. This can now be easily automated or controlled by non-BizTalk experts.

Monitoring

Most new opportunities reside on the monitoring side. A couple of potential use cases:

  • Simplified and short-lived BAM. It's possible to create some simple reports with basic statistics of your BizTalk environment. You can leverage the Management API or the Operational OData Service. You can easily visualize the number of messages per port and for example the number of suspended instances. All of this is built on top of the data in your MessageBox and DTA database, so there's no long term reporting out-of-the-box.

  • Troubleshooting. There are very easy-to-use operations available to get a list of services instances with a specific status. In that way, you can easily create a dashboard that gives an overview of all instances that require intervention. Suspended instances can be resumed and terminated through the Management API, without the need to access your BizTalk Server.

This is an example of the basic Power BI reports that are shipped with this feature pack.

What is not included?

This brand new BizTalk Management API is quite complete, very excited about the result! As always, I looked at it with a critical mindset and tried to identify missing elements that would enable even more additional value. Here are some aspects that are currently not exposed by the API, but would be handy in future releases:

  • Host Instances: it would be great to have the opportunity to also check the state of the host instances and to even start / stop / restart them. Currently, only a GET operation on the hosts is available.

  • Tracked Context Properties: I'm quite fond of these, as they enable you to search for particular message events, based on functional search criteria (e.g. OrderId, Domain…). Would be a nice addition to this API!

  • Real deployment: first I thought that the new deployment feature was built on top of this API, but that was wrong. The API exposes functionality to create and manage ports, but no real option to update / deploy a schema, pipeline, orchestration or map. Could be nice to have, but on the other hand, we have a new deployment feature of which we need to take advantage of!

  • Business Activity Monitoring: I really like to idea of the Operational OData Service, which smoothly integrates with Power BI. Would be great to have a similar and generic approach for BAM, so we can easily consume the business data without creating custom dashboards. The old BAM portal is really no option anymore nowadays. You can vote here.

Conclusion!

Very happy to see more commitment from Microsoft towards BizTalk Server. This emphasises their "better together" integration vision on BizTalk Server and Logic Apps! Check out the BizTalk User Voice page if you want to influence the BizTalk roadmap!

The exposure of BizTalk as a REST API opens up a new range of great opportunities. Don't forget to apply the required security measures when exposing this API! By introducing this API, the need for auditing all activity becomes even more important!

Thanks BizTalk for this great addition! Thank you for reading!

Cheers,
Toon

Categories: BizTalk
written by: Toon Vanhoutte

Posted on Wednesday, May 3, 2017 5:06 PM

Toon Vanhoutte by Toon Vanhoutte

Recently, the product team released a first feature pack for BizTalk Server 2016. Via this way, Microsoft aims to provide more agility into the release model of BizTalk Server. The feature pack contains a lot of new and interesting features, of which the new scheduling capabilities is an important one. Let's have a look at what's included!

The documentation of this scheduling feature can be found on MSDN.

What is included?

Support for time zones

The times provided within the schedule tab of receive locations are now accompanied by a time zone. This ensures your solution is not depending anymore on the local computer settings. There's also a checkbox to automatically adjust for daylight saving time.

This is a small, but handy addition to the product! It avoids unpleasant surprises when rolling out your BizTalk solutions throughout multiple environments or even multiple customers!

Service window recurrence

The configuration of service windows is now a lot more advanced. You have multiple recurrence options available:

  • Daily: used to run the receive location every x number of days
  • Weekly: used to run the receive location on specific days of the week
  • Monthly: used to run the receive location on specific dates or specific days of the month

Up till now, I didn't use the service window that much. These new capabilities allow some new scenarios. As an example, this would come in handy to schedule the release of batch messages on a specific time of the day, which is often required in EDI scenarios!

What is not included?

This is not a replacement for the BizTalk Scheduled Task Adapter, which is a great community adapter! There is a fundamental difference between an advanced service window configuration and the Scheduled Task Adapter. A service window configures the time on which a receive locations is active, whereas the Scheduled Task Adapter executes a pre-defined task on the configured recurrence cadence.

For the following scenarios, we still need the Scheduled Task Adapter:

  • Send a specific message every x seconds / minutes.
  • Trigger a process every x seconds / minutes.
  • Poll a rest endpoint every x seconds / minutes. Read more about it here.

Conclusion!

Very happy to see more commitment from Microsoft towards BizTalk Server. This emphasises their "better together" integration vision on BizTalk Server and Logic Apps! Check out the BizTalk User Voice page if you want to influence the BizTalk roadmap!

These new scheduling capabilities are a nice addition to BizTalk's toolbelt! In future feature packs, I hope to see similar capabilities as the Scheduled Task Adapter. Many customers are still reluctant to use community adapters, so a supported adapter would be very nice! You can vote here!

Thanks for reading!
Toon

Categories: BizTalk
written by: Toon Vanhoutte

Posted on Wednesday, December 21, 2016 9:35 AM

Pieter Vandenheede by Pieter Vandenheede

On December 13th I spoke at BTUG.be XL about BizTalk 2016, the new features and the key aspects of its Always On support. In this post I share with you my slide deck.

Last week, I had a great time at BTUG.be, while presenting my session on BizTalk 2016.

I presented the new features in BizTalk Server 2016 RTM and a few takeaway from SQL Server 2016. More specifically, and in-depth, on SQL Server AlwaysOn support for BizTalk Server 2016 on-premise and in the Azure cloud, as well as an intro on the new Logic App adapter and how to install and connect it to your on-premise BizTalk Server.

As promised there, please find my slide deck below via SlideShare:

Contact me if you have any questions regarding the slides, I'd be happy to answer you.

The other speakers there were Glenn Colpaert (session about Azure Functions), Kristof Rennen (session on Building scalable and resilient solutions using messaging) and Nino Crudele (session on Holistic approaches to Integration).

As always, it was nice to talk to the people present. A big thank you to BTUG.be for having me again!

Enjoy the slide deck!

Pieter

Categories: Community
written by: Pieter Vandenheede

Posted on Tuesday, November 22, 2016 5:14 PM

Pim Simons by Pim Simons

Why do HTTP 400 status codes return a NACK, while HTTP 500 status codes return an ACK? And how to deal with this...
This post describes an interesting scenario we encountered at a customer and our considerations for dealing with it.

At one of our customers I had implemented the ReturnAddress messaging pattern (http://www.enterpriseintegrationpatterns.com/patterns/messaging/ReturnAddress.html), by using a generic BizTalk application which sends an asynchronous response message to a client application. The solution had been running successfully for some time, when we encountered a strange situation.

The BizTalk application uses a one-way WCF-Custom send port, using a wsHttpBinding, to send a message to the client application. Also, I had added the Delivery Notification functionality to make sure messages are delivered successfully.

It is important to realize that one-way send ports that use SOAP will receive a technical response containing an HTTP status code. If and when the send ports receive a HTTP status code in the 200 range, the Delivery Notification generates an ACK and BizTalk knows the message was successfully delivered.

So far, so good. The application had been through testing on the Test and Acceptance environments, had been deployed to the Production environment and had been running for several months without any problems. Until it appeared that some of the messages that were being sent using the generic BizTalk application were ‘not arriving’ at the client application. This would happen at random and, what was really strange, the logging in BizTalk showed that the message was successfully sent and BizTalk had received an ACK response as part of the Delivery Notification. Also there was no mention of an error in the event log of the BizTalk servers.

After some debugging we found the source of the problem. The message sent by BizTalk was successfully received by the client application, however the client application encountered an error processing the message and returned the HTTP 500 status code. So now the question was, why is the Delivery Notification not generating a NACK response when a HTTP 500 status code is received? I had expected that any status code in the HTTP 400 and 500 range would result in a NACK.

This turned out not to be the case. While status codes in the HTTP 400 range will result in a NACK, the status codes in the HTTP 500 range will result in an ACK and BizTalk will view this message as successfully delivered at the client application. The logic behind this seems to be that the status codes in the HTTP 400 range indicate that the message was not received by the client application (hence the NACK) and the status codes in the HTTP 500 range indicate that the message was received by the client application, but that the client application encountered an exception. Since the message was delivered at the client application, BizTalk views this as a successful delivery and will generate an ACK as part of the Delivery Notification.

Unfortunately, there isn’t any documentation on MSDN on which status codes will return an ACK or NACK.

The documentation on the SOAP HTTP response states that “In case of a SOAP error while processing the request, the SOAP HTTP server MUST issue an HTTP 500 "Internal Server Error" response and include a SOAP message in the response containing a SOAP Fault element indicating the SOAP processing error”. For reference, see https://www.w3.org/TR/2000/NOTE-SOAP-20000508/#_Toc478383529.

Some discussion followed on the validity of catching the HTTP 500 error in BizTalk, since the message was successfully delivered and accepted by the client application. That means that, from a technical perspective, the responsibility would now lie at the client application to handle the error. From a functional responsibility perspective however, it was decided to find a way to catch the HTTP 500 error in BizTalk, as this would enable the customer's administrators to use the same resubmit functionality we had created by using a generic BizTalk error handling framework.

So I had to make sure the HTTP 500 status code was somehow caught, so that BizTalk would return a NACK which would result in the error handling catching the error. Fortunately, this can be achieved quite easily by implementing a WCF behavior on the one-way send port. The WCF behavior checks in the AfterReceiveReply message inspector if the reply is a fault message, and if so it will throw an exception using the fault description.

By implementing this WCF behavior on a one-way send port BizTalk will generate a NACK when a response is received with an HTTP status code in the 400 or 500 range. Sometimes the default behavior surrounding technical responsibility doesn’t align with the requirements and responsibilities from a functional point a view, and this may just offer a solution for you as well.

Categories: BizTalk
written by: Pim Simons

Posted on Monday, August 17, 2015 12:00 AM

Pim Simons by Pim Simons

I was recently asked to assist one of our customers with a problem they were experiencing with one of their BizTalk interfaces. They had created an orchestration that was sending a message using a logical port set to “Specify Later”, which was bound to a physical Send Port Group containing multiple Send Ports.

The error message I was faced with was:

“The instance completed without consuming all of its messages. The instance and its unconsumed messages have been suspended.”

After checking the orchestration I quickly found the cause of the error: The Delivery Notification setting of the logical port in the orchestration had been set to “Transmitted”. This meant that after sending the message, the orchestration expects exactly one (N)ACK message back to determine whether the message had been received successfully. Because a Send Port Group with multiple Send Ports was being used, multiple (N)ACK message were being received (one for each Send Port) while only a single (N)ACK was expected by BizTalk. Since, after the first message is received, there are no instances/subscribers remaining for the other incoming (N)ACK responses, this resulted in zombie messages.

Obviously the first thing I tried was searching for a solution online, hoping that someone had already solved this problem. After coming across a discussion that said that this was impossible I just had to see for myself.

One of the ways to approach this issue would be by simply using multiple send shapes in the orchestration and stop using the Send Port Group all together. In this case this was not an option however, since one of the requirements for this interface was that it should be very easy to add and remove locations where the file would be send to, as destinations for the file change regularly. The administrators in charge of maintaining the interface preferred to keep using the Send Port Group instead of having to change and redeploy the orchestration for every new Send Port. At the same time, receiving the delivery notification for each outgoing message was also a key requirement.

So I had to come up with an approach in which I would cover all requirements, which basically meant that I had to keep using the Send Port Group. This meant that I could not use the out-of-the box Delivery Notification setting on the logical port in the orchestration, so we had to recreate this functionality. Fortunately this specific part had been done before and is described in this excellent blogpost by Bruno Câmara: http://www.agilior.pt/blogs/bruno.camara/archive/2008/04/17/4344.aspx.

In short, this is done by setting the promoted property BTS.AckRequired on the outgoing message to true, initializing the BTS.CorrelationToken to new guid and creating receive shapes for the (N)ACK message. By using correlation sets for these promoted properties the (N)ACK message is received in the orchestration. After receiving the (N)ACK message in the orchestration we can check if it’s an ACK or NACK message and handle it accordingly.

However, simply recreating the Delivery Notification functionality would not be enough; we had to make sure the orchestration would receive every Delivery Notification message. To do this I used the Delivery Notification functionality in a loop. I created a function to retrieve the number of Send Ports currently present and bound in the Send Port Group using WMI to determine the amount of iterations for the loop.

The final solution in the orchestration ended up looking like this:

The orchestration sends the Output message, which is then picked up by the Send Port Group and sent to any number of locations. The orchestration uses a custom function to determine the number of Send Ports in the Send Port Group and uses this as a parameter in the Loop expression. In the Loop the Delivery Notification messages are received, if a NACK message is received the error message is logged.

After running the interface, the Tracked Message Events for the orchestration now show that multiple Delivery Notifications have been received:

By using this solution we only had to modify and redeploy the interface once, leaving the administrators with the ability to add and remove Send Ports to and from the Send Port Group without the need to go into Visual Studio and deploying a new version of the orchestration, all while avoiding zombie messages!

And finally, here is the code used to retrieve the number of Send Ports in a Send Port Group:

Categories: BizTalk
written by: Pim Simons