wiki

Codit Wiki

Loading information... Please wait.

Codit Blog

Posted on Thursday, March 10, 2016 3:38 PM

Massimo Crippa by Massimo Crippa

With the latest Azure API Management service update, the Git integration has been rolled out. In this post we will see how we can seamlessly control the versions of the proxy configuration and move it between different environments.

Scenario

All the configuration that are applied to the proxy and the portal customizations are stored in a database in the provisioned tenant. Now, every APIM tenants can expose a public Git endpoint to which we can refer to PULL the configuration down to our local (or remote) Git repository.

Once we have our local version, we can apply the changes we need, push it back to the tenant repository and then save it to the APIM database.

In the diagram below, the steps of our scenario

  1. Save (sync from the APIM repository to Tenant Git)
  2. Pull (From Tenant Git to local repo),
  3. Apply changes (on our local repo),
  4. Push (to Tenant Git)
  5. Deploy (from the Tenant Git to the APIM repository)

 

The Git integration is not enabled by default, so first we have to connect to the management portal, go to the Security area and enable the Git access. 

Save and Pull

Next step is to save the proxy configuration to the APIM tenant’s Git repository. This can operation can be done in two different ways:

  • Use the “Save Configuration to Repository” button on the APIM Admin Portal.
  • Call the “Save” operation of the API Management REST API (here how to enable the REST API).

In both cases you have to specify the branch name where to save the configuration and whether to override or not the changes in newer check-ins. This operation can takes a couple of minutes.

Once completed, you can open a Git console and pull the configuration to create a working copy of the remote repository by using the clone command.

Before that you need to get (in the Admin Portal) a temporary password to access to the remote repository.

 

Then run the “git clone https://coditapi.scm.azure-api.net/” command and specify “apim” as username and the temporary password we got at the step before.

Below the folder structure of the local repository. As you can see the proxy configuration is exported (apis, policies, security groups and products) along with the developer portal customizations.

If in the meanwhile a new configuration has been saved in the APIM repo, we can pull it down with the "git pull" command.

Apply a change

Let's imagine we want to change the policy applied to the Echo API to extend the existing basic round robin algorithm.

The policy is applied at API scope so the file to be edited is policies/apis/Echo_API.xml 

This is the result of the "git diff" command after the change.

Now, in order to add the change to the git staging area use the “git add -A” command and then commit the changes with "git commit -m" as in the picture below.

Now we’re ready to push our changes to the Azure API Management Git repo.

Push and deploy

Type “git push” to sync the changes with the repository on our API Management tenant.

The final step is to deploy our configuration from the tenant repository to the APIM proxy.

This can operation can be done in two different ways:

  • Use the “Deploy Repository Configuration” button on the APIM Admin Portal.
  • Call the “Deploy” operation of the API Management REST API (here how to enable the REST API).

For this step I'm going to invoke the Configuration REST API using postman.  Here the details of my API call.

Method : POST
Address : http://{tenantname}.management.azure-api.net/configuration/deploy?api-version=2014-02-14-preview
Headers :
 + Content-Type > application/json
 + Authorization > SharedAccessSignature=....
Body :
    {"branch":"master"}

 

As response I got a 202 (Accepted) and the Location header with the link to the check the status of this transaction.

With the operationResults operation to check the status (Failed, InProgress, Succeded) of the deploy. It's a GET and again we must specify the Authorization header as in the previous call.

If the deploy succeeded, the changes are immediately applied to the proxy. 

Restore a configuration

Now, imagine that you've applied a wrong configuration to the proxy and you want to restore a previous version from your local git repository. For example, these are the timestamps of my configurations:

  • On proxy: updated at 10:22 AM
  • On Teneant Repo: updated at 10:18 AM
  • Local repo: updated at 08:19 AM

I want to discard the 10:18 AM version and replace the 10:22 AM version with the 08:19 AM one. It's a four step procedure. 

A) First thing to do is to bring the tenant repo in sync with the proxy. This step is necessary to mark the proxy as synched. Without the sync you will get this error as result of the the deploy operation : "Deployment operation failed due to invalid data: Snapshot operation is not safe.  Latest sync date: '2015-11-10T10:18:35.5278745'; latest configuration update date: '2015-11-10T10:22:01.7635694'"

B) Apply a modification to the local repo and commit it. This is necessary so the deploy procedure can recognize that there is something to be overwritten. 

C) run the "git push -f origin master" command to overwrite the version in the tenant git.

D) deploy the configuration using the Admin Portal or via REST API

Conclusion

The git integration is a feature that customers have been asking for a while. Now, you can create and manage different versions of your proxy configurations and move them between different environments.

Cheers,

Massimo

Categories: Azure
written by: Massimo Crippa

Posted on Saturday, February 27, 2016 12:00 AM

Luis Delgado by Luis Delgado

Micro-services architectures are gaining popularity as a software architectural pattern. There are many aspects to think about when considering a micro-services architecture: scalability is one of them. Let's contrast scalability with common alternatives.

Vertical scalability

With vertical scalability, you scale the capacity of your application by increasing hardware capacity. This is why it is named "vertical": you add more CPU, more memory, more disk IOPS, but effectively, the architecture of your app and infrastructure does not change. This is a viable scalability pattern but it has an obvious hard-stop: there is so much hardware you can add, up to a certain point.

Horizontal scalability

With horizontal scalability, instead of scaling up by adding more hardware capacity, you architect your application so that it scales out by adding more instances of it. This can be accomplished by adding more VMs with the application installed, more application instances inside a cloud service, or more containers... you get the idea. You don't need beefy, expensive hardware for horizontal scalability, you can get along with small machines, and add many of them. This scalability pattern usually requires adjustments in the application architecture. For example, since a client request may be served by any machine (out of many), the application typically has to be stateless, or if state is needed, it needs to be stored somewhere else.

Scalability with micro-services

Whereas the concept of scaling horizontal seems appealing, remember that every instance you "clone" to scale horizontally is running a complete instance of your application. This might be undesirable, as your application might have different scalability needs. Typically, the load of an application is not evenly-distributed among all the services it provides. For example, a careful analysis of telemetry data might show that the bottleneck in your application are the authentication services, but all other services inside your app are performing well. If you scale horizontally, you will be scaling out the authentication services... along with everything else that does not need to be scaled out. This is a waste of resources.

A micro-services architecture takes an application and splits it into independent, working, functional units, called "services". Don't get mislead by the word "micro" in "micro-services". The split of services does not need to be "microscopic". You can split the services within your application in any arbitrary way you want. Typically, the more atomic the services are, the more value you will get from this architecture, but it need not be the case every time.

With that out of the way, let's go back to our example. With a micro-services architecture, your app will run different services as independent units, each with its own runtime, codebase, processing thread(s), etc. Since the bottleneck in our app is the authentication routine, you can scale out that service only, and leave the rest alone. With micro-services, you make a more effective use of the horizontal scalability pattern.

When considering a micro-services architecture, there are many more factors to analyse beyond scalability. But mixing the micro-services architecture with horizontal scalability typically gives you better capacity elasticity than using a monolithic architecture.

Categories: Architecture
Tags: Scalability
written by: Luis Delgado

Posted on Thursday, February 18, 2016 3:56 PM

Michel Pauwels by Michel Pauwels

Short blog post on Microsoft BizTalk Server incorrectly identifying an issue with the BizTalk FTP Adapter login.

One of our customers had just enabled a new FTP receive location after configuring it and kept getting "The username or password is incorrect" warnings in the Event Viewer which caused the receive location to disable after a while.

The obvious course of action was to re-enter the password and try again. After the obligatory turning it off and on again of both the Receive Location and even the Host Instances, the error remained.

We then enabled the logging on the FTP Receive Location (file logging feature) and after the next recycle we could see that the log contained the following lines:

< 220 TYPSoft FTP Server 1.11 ready...
> USER xxxx
< 331 Password required for MyUserName.
> PASS xxxx
< 530 Too Many Users Connected.

Even though the password is correct, BizTalk insisted the username or password was incorrect, presumably because it got a 530 code after it sent the PASS command to the server and assumed it was due to a login failure.

In short, what we learned today was that it's a good idea to enable the FTP logging when you're getting username / password failures, because the error could lead you to wrong conclusions.

Categories: BizTalk
written by: Michel Pauwels

Posted on Friday, February 5, 2016 12:50 PM

Toon Vanhoutte by Toon Vanhoutte

This blog post series aims to provide an introduction to AS4, a recent B2B messaging standard. It provides the basic insights on how AS4 establishes a reliable and secure message exchange between trading partners. This is a jump start to get to know the ins and outs of the standard, as the specifications are quite complex to understand without any prior introduction.

If you are active in B2B/B2C messaging, then AS4 is definitely coming your way! Make sure you are prepared.

AS2 Comparison

As AS4 is inspired by AS2, both are often considered in message exchange over the internet. On a high level, they are very similar, however there are some key differences that you should be aware of in order to make a profound decision on this matter. Let’s have a look!

Common Characteristics

These are the most important common characteristics of AS2 and AS4:

  • Payload agnostic: both messaging protocols are payload agnostic, so they support any kind of payload to be exchanged: XML, flat file, EDI, HL7, PDF, binary…
  • Payload compression: AS2 and AS4 support both compression of the exchanged messages, in order to reduce bandwidth. It’s however done via totally different algorithms.
  • Signing and encryption: the two protocols support both signing and encryption of the exchanged payloads. It’s a trading partner agreement whether to apply it or not.
  • Non-repudiation: the biggest similarity is the way non-repudiation of origin and receipt are achieved. This is done by applying signing and using acknowledgement mechanisms.

Technology Differences

The common characteristics are established by using totally different technologies:

  • Message packaging: within AS2, the message packaging is purely MIME based. In AS4, this is governed by SOAP with Attachments, a combination of MIME and SOAP.
  • Security: AS2 applies security via the S/MIME specifications, whereas AS4’s security model is based on the well-known WS-Security standard.
  • Acknowledgements: in AS2 and AS4, acknowledgements are introduced to support reliable messaging and non-repudiation of receipt. In AS2 this is done by so-called Message Disposition Notifications (MDN), AS4 uses signed Receipts.

AS4 Differentiators

These are the main reasons why AS4 could be chosen, instead of AS2. If none of these features are applicable to your scenario, AS2 might be the best option as it is currently more known and widely adopted.

  • Support for multiple payloads: AS4 has full support for exchanging more than one business payload. Custom key/value properties for each payload are available.
  • Support for native web services: being built on top of SOAP with Attachments, AS4 offers native web service support. It’s a drawback that SwA is not supported out-of-the-box by .NET.
  • Support for pulling: this feature is very important if message exchange is required with a trading partner that cannot offer high availability or static addressing.
  • Support for lightweight client implementations: three conformance clauses are defined within AS4, so client applications must not support immediately the full AS4 feature stack.
  • Support for modern crypto algorithms: in case data protection is an important aspect, AS4 can offer more modern and less vulnerable crypto algorithms.
  • Support for more authentication types: AS4 supports username-password and X.509 authentication. There is also a conformance clause on SAML authentication within AS4.

Getting Started

Are you interested to learn more on AS4?

From an architecture / specifications perspective it’s good to have a look at the related OASIS standards. This includes the ebMS 3.0 Core Specifications and the AS4 profile of ebMS 3.0. In case you are more interested on how an AS4 usage profile could be described between trading partners, this ENTSOG AS4 Usage Profile is a great example.

If you’re more developer oriented, it’s good to have a look at Holodeck B2B. It is a java-based open-source software that fully supports the AS4 profile. Some sample files provide you a head start in creating your first AS4 messages. Unfortunately SOAP with Attachments or AS4 is not supported out-of-the-box within Microsoft .NET.

Can we help you?

Codit is closely involved with AS4. It is represented in the OASIS ebXML Messaging Services TC by Toon Vanhoutte, as a contributing member. In this way, Codit keeps a close eye on the evolution of the AS4 messaging standard. Throughout several projects, Codit has gained an extended expertise in AS4. Do not hesitate to contact us if you need any assistance on architecture, analysis or development of your AS4 implementation.

Within our R&D department, we have developed a base .NET library with support for the main AS4 features. In the demo for Integration Monday, this library was plugged into the BizTalk Server engine in order to create AS4 compatible messages. At the time of writing, Codit is defining its AS4 offering. If you would like to be informed about the strategy of Codit when it comes to AS4, please reach out to us. We will be more than happy to exchange ideas.

Categories: Technology
Tags: AS4
written by: Toon Vanhoutte

Posted on Thursday, February 4, 2016 4:39 PM

Toon Vanhoutte by Toon Vanhoutte

This blog post series aims to provide an introduction to AS4, a recent B2B messaging standard. It provides the basic insights on how AS4 establishes a reliable and secure message exchange between trading partners. This is a jump start to get to know the ins and outs of the standard, as the specifications are quite complex to understand without any prior introduction.

Security is a crucial aspect of every messaging standard, so it is for AS4! This section describes the most common mechanisms to establish a secure AS4 communication.

Non-repudiation of origin

In order to guarantee message integrity, it’s important to have non-repudiation of origin. This means that the receiver is a 100% sure that the message originates from the sending party and that the message was unmodified during the exchange.

AS4 provides this evidence by applying the WS-Security signing feature on User Messages. This message signing is performed via asymmetric keys: the sender of the message signs with its own private key and the receiver authenticates the sender via the public key, which could be included in the message as a BinarySecurityToken.

According to the AS4 usage profile, the following parts of the ebMS message must be signed:

  • The ebMS Messaging Header
  • The SOAP Body
  • All SOAP Attachments

The receiving MSH checks if the hash values within the signature correspond to the actual received payloads. If not, an ebMS Error is generated and sent to the sending MSH, according to the agreed P-Mode configuration.

Non-repudiation of receipt

Another important aspect is non-repudiation of receipt. This means that the sender is a 100% sure that receiver has received the message, without being modified during the message exchange.

AS4 provides legal proof for this non-repudiation of receipt, by applying the WS-Security signing feature on Receipts. The exchanged Receipts are signed by the receiving MSH, which authenticates the originator of the receipt. The signature of Receipts must be applied on the ebMS Messaging Header.

In addition to this signature, the ebMS Messaging Header must include Non Repudiation Information within the Receipt element. This Non Repudiation Information contains the hashes of the signed payloads of the referenced User Message. This information is verified again at the sending MSH, in order to establish evidence for non-repudiation of receipt.

Data confidentiality

Data confidentiality can be ensured on both the communication and message level.

Encryption, which ensures data confidentiality, on the communication channel could be applied by using Transport Layer Security (SSL). Due to known vulnerabilities in this HTTPS protocol, it is recommended to use TLS 1.1 or higher. TLS may be offloaded to specific infrastructure components (e.g. proxy), so an AS4 MSH does not need to explicitly offer this functionality.

Data confidentiality on the message level could be achieved by leveraging the WS-Security encryption feature on User Messages. This message encryption is performed via asymmetric keys: the sender of the message encrypts with the public key of the receiver, so the message can only be decrypted by the receiver, who owns the private key. Encryption must be applied on:

  • The SOAP Body
  • All SOAP Attachments

In case the receiving MSH is unable to decrypt the message, an ebMS Error is generated and sent to the sending MSH, according to the agreed P-Mode configuration.

 

Categories: Technology
Tags: AS4
written by: Toon Vanhoutte