wiki

Codit Wiki

Loading information... Please wait.

Codit Blog

Posted on Wednesday, May 27, 2015 4:14 PM

Tom Kerkhove by Tom Kerkhove

Microsoft's Yammer has been around for a while and people who are part of one or more networks will agree that Yammer can turn into Spammer.

In this blog post I demonstrate how you can automatically post to a Slack channel.

This blog post was also released on my personal blog.

Microsoft's Yammer has been around for a while and people who are part of one or more networks will agree that Yammer can turn into Spammer.

For each new conversation & comment, Yammer will send you an email resulting in mail floods. The easy fix would be to disable the notification email but then you risk the chance to miss out on interesting/important discussions.

At our current project we use Slack to communicate with each other and it's a really nice tool - Nice & clean just how I like it.

Slack Logo

So lets get rid of the notification emails and notify your team when someone starts a new conversation on Yammer! This is where Microsoft Azure App Services come in, more specifically Microsoft Azure API & Logic Apps.

With Azure Logic apps I've created a flow where I have one API app listening on a Yammer group for new conversations while another Slack API App will notify us in a channel when something pops-up.

How does that look?!

When I create a new conversation in Yammer i.e. "We're ready to go in production" - 
New Yammer Conversation

The Yammer API App in my Logic App will notice that there is a new conversation and will send a message to my team's Slack channel as the Project Announcements-bot. 

Slack Bot Response

Want it yourself? Here's how!

Before getting our hands dirty let's summarize what's on today's schedule.

We will start with provisioning our API apps that we will use from the Azure Marketplace. After that we will create a new Logic app that will describe the flow of our app.

Provisioning the API Apps

As of today you have two options for provisioning your API Apps - One is to provision them upfront where you have more control on naming and such. Second is provision them while you are designing your Logic app and let Azure take care of the naming. Be aware: Azure uses names like YammerConnector1431593752292 that doesn't really say where they're being used.

Since I always want to name my components as self-describing as possible we will provision two API apps up front :

  • A Yammer App that will trigger our Logic app when a new conversation is posted
  • A Slack App that will send a message to Slack as a Bot

Provisioning an API App is super simple : Browse to the new Azure Portal > Click New > Select Web + Mobile > Browse the Azure Marketplace > Select the API Apps section > Select the API App you want.

After you've selected your API App you basically you give it a Name, assign the App Plan & Resource Group : 
Provision API App

Azure will start provisioning the API App for you in the background, while they are doing that let's have a look at the Connector Info.

Before actually provisioning the App you see that each API App or Connector gives you an overview of it's capabilities in a Logic App. Here you can see that the Slack Connector will only be able to act in a Logic app. 
Slack Connector overview

Now when we look at the Yammer Connector Info we see that it can act withing a Logic App but also Trigger it on a certain condition. 
Yammer Connector overview

Defining the flow in a Logic App

Before we can start defining our flow we need to create a new Logic App.

In the Azure Portal click New > Select Web + Mobile > Logic App. Give it a self-describing name and add it to the same App Plan as your provisioned API Apps.

Once it is configured, open it and click Triggers and Actions
Configure Logic App

We will define our flow by defining the sequence of connectors. You can find our provisioned connectors on the side, click on your Yammer connector to add it. 
Clean Logic App

After that, the default card will be replaced with your Yammer connector. As you can see we first need to authenticate with Yammer. Click Authorize
Basic Yammer Card

A pop-up will show to do the OAuth dancing with Yammer. After you've logged in you will see need to grant access to your Logic App.

Read the statement carefully and click Allow if you agree.

Yammer Authentication

(In order to complete the following steps you need to allow access)

Now that you've allowed access to your Yammer account it's interesting to know that the authentication token will be stored in the secure store of the Gateway (A Gateway is used by API Connectors to communicate with each other and outbound services). This is because the gateway will handle all the authentication with Yammer for us.

Once that's done you get an overview of all the triggers the Yammer connector has. Luckily the only one that is available is the one we need, click New Message
Yammer Triggers

Configuring the trigger is fairly easy - We define the trigger frequency in which the connector will look for new messages. Next to that we assign the Group Id of our Yammer Group that we are interested in. The granularity of your trigger frequency depends on the hosting App Plan. In my example I'm using 1 minute which requires me to use a Standard-tier App Plan.

You can find the group Id by browsing to your group and copying the Feed Id.

https://www.yammer.com/Your-Network/#/threads/inGroup?type=in_group&feedId=579250

Yammer Connector Configured

Click the checkmark to save your configuration.

Go back to the side bar and click on your Slack Connector to add it to the pane. Here we need to authenticate with our Slack by clicking Authorize
Basic Slack Connector

Just like with Yammer, Azure will request access to your Slack account to post messages. 
Slack Authentication

Our last step is to configure the Slack connector.

What we will do is send the original message as a quote along with who posted it and a link to the conversation. In Slack that results in the following markup statement -

>>> _"Original-Message"_ by *User* _(Url)_

To achieve this we will use the @concat function to assign the Text value -

This statement is retrieving some of the output values of the Yammer connector.

We will also configure to which Slack channel you want to send it. Optionally you can assign a name to the Slack bot and give it a icon. Here I gave the name of my Yammer group as Slack bot name. 
Slack Connector Configured

Click the checkmark to save your configuration & save the flow of your logic app. 
Save Logic App

After a few seconds/minutes, depending on your trigger configuration, you will see that the Yammer connector picked up your new message and triggered your Logic App. 
Logic App - Run Overview

Now you should see a new message in your Slack channel!

Ship it!

That's it - we're done!

Your Yammer connector will now poll for new conversations in your Yammer group every cycle you've defined in its configuration. If there are new ones, your Logic App will start processing it and you will be notified in Slack!

Wrapping up

As you can see, you can very easily use Azure API & Logic Apps to create small IFTT-like flows. Nevertheless you can even build more full-blown integration scenarios by using the more advanced BizTalk API Apps!

If you want you can even expand this demo and add support for multiple Yammer groups. To do so you'll need to open the Code View and copy additional triggers in the JSON file (Thank you Sam Vanhoutte for the tip on how to create multiple trigger).

Keep in mind that the Slack bot's name that is posting is currently hardcoded, unfortunately the Yammer app doesn't expose the name of the group so this is something you'll have to work around.

Can't get enough of this? You can build your own API App or read Sam Vanhoutte his initial thoughts on Azure App Services!

Thanks for reading,

Tom Kerkhove

Categories: Azure App Services
written by: Tom Kerkhove

Posted on Wednesday, May 13, 2015 7:28 PM

Maxim Braekman by Maxim Braekman

Sam Neirinck by Sam Neirinck

Tom Kerkhove by Tom Kerkhove

The second edition of Techorama, which is being hosted at Utopolis Mechelen, provided a large range of interesting sessions covering all kind of topics. Read more about some of the sessions from the second day in this post.

Just as promised in yesterday’s post of day 1, we are back with an overview of some of the sessions from the second day of Techorama.

Internet of things, success or failure by Stefan Daugaard Poulsen

Twitter: @cyberzeddk

One of the sessions to start off the second day of Techorama, was one about the internet of things, presented by Stefan. He made it clear from the very beginning, that this was not going to be a technical session about writing the code to run on devices, nor about the electronics themselves since Stefan, to put it in his own words, knows jack about it.

Companies are continuously attempting to invent new devices for all kinds of purposes, but are all of these devices actually useful? It’s not all about inventing shiny new devices that look good, but they should take some aspects into account:

  • Does it solve a problem? Can it be used to actually make life easier or provide useful information.
  • Is it consumer-friendly? In other words, can it be shipped without user-manual without raising questions.
  • Does it repeat history? There is no use in re-creating devices that clearly failed in the past.

Off course, one could ask a whole bunch of other questions before starting the development or creating a kickstarter-project. But these questions above are vital in order to build a device, which might turn into a succes.

Although the Internet of Things is becoming widely popular and lots of companies are jumping onto the IoT-train, there are quite some challenges:

  • Privacy: what happens with the data that is being collected by this device.
  • Security: since most devices will be connected to a network, they may not become the culprit of security-leaks.
  • Data processing: all of the sensors are generating a huge load of data, which needs to be processed in an orderly way.
  • Data storage: all of this data that is being processed needs to be stored in a correct way. Do you actually need of the data? How long do you need to save it?
  • Futuristic thinking: the devices should be an enhancement of the current world, but with some limitations. It is not always possible to change how everything is currently working, without expensive modifications.
  • Battery-life: there is no use in creating a device that needs to be charged every couple of hours.

In overall, people or companies should think before creating the next new thing, as it needs to be useful, non-intrusive, reliable and enhancing.

Embellishing APIs with Code Analyzers by Justin Rusbatch

Twitter: @jrusbatch

Visual Studio 2015 ships with the long-awaited Roslyn compiler platform. I can’t remember when exactly Microsoft started talking about Compiler as a Service, but it’s been a couple of years at least. However, it was worth the wait!

As is more and more common within Microsoft, the development of this platform is all open on Github. This means the compiler is no longer a black box, but a fully featured set of APIs which can be used to analyze code, among many other things.

Justin did a small demo on how easy it was to create and debug an Analyzer using Visual Studio 2015 and the VS2015 SDK. It was a simple demo analyzer which would indicate that class names must be in upper case (I suggest not to use this in your actual code).

A code analyzer looks like this in Visual Studio: 
enter image description here

I can think of quite a few use cases already to use code analyzers for. If we think about BizTalk development alone, one can imagine quite a few rules to create, just for Pipeline Components.

  • The pipeline component class must have a GuidAttribute and ComponentCategoryAttribute. (this prevents a few minutes of wondering why your pipeline component doesn’t show up in the Toolbox).
  • Do in-depth code analysis to see if the Load&Save methods are implemented correctly.
  • Create warnings for working with non-streaming classes.

Additionally, each integration project has business-specific rules and coding/naming guidelines. Perhaps your guidelines require you to do a LogStartMethod() & LogEndMethod() in each and every method. Now you can create an analyzer which can verify this, and optionally break your build. This way you can ensure that all your guidelines are enforced, and you have great Visual Studio tooling as an additional benefit. You can even create quick fixes so it’s just a matter of clicking the light bulb and the log statements are inserted without you typing a thing.

All in all, it’s something I will definitely look into in the coming weeks.

Teamwork - Playing Well With Others by Mike Wood

Twitter: @mikewo

Those who've read yesterdays post already know I'm a fan of Mike as a speaker but todays session was really really inspiring!

The focus of the talk was about how can you as an individual work well with others - The first step to achieve this is by stop thinking about yourself as an individual, but instead think the team. Together you are one and believe in one big picture - Your main objective. You need to get rid of your ego and work together as a team to achieve your goal and get across the hurdles that are holding your back from achieving it.

Here are some other interesting tips he gave us :

  • When communicating in team, do it wisely - Don't start pointing fingers at each other when things fail but work together to fix it as soon as possible. Talk in the we form when it's possitive, otherwise talk in the I form, i.e. I broke the build because of this or that. Avoid the "Lottery"-effect where only one person has knowledge about a certain topic, losing him/her means losing a lot of knowledge.

  • Great work can be rewarded with incentives but do it the right way - Reward the team instead of one individual. As an example don't reward the salesman who sold the most, reward the team when they've reached a certain target. This will boot their team spirit instead of having internal competition.

  • Understand failure and accept it - Nobody is perfect & everybody makes mistakes so accept this, it's inevitable to make mistakes but make sure you learn from them.

  • Leadership - Not everyone wants to be a leader so don't push people to do this. A true leader knows how his team members work & feel so they can take that into account. Provide them with guidance & your vision to which you are striving. Also delegation is key to success but don't assign them tasks you would not want to do on your own.

  • Invest in your team members - Have trust in them and let them research things they're interested in

These are just some of the examples Mike gave us that can really contribute in thinking as a team, working as a team & shipping great solutions as a team.

I'd like to end this blog post with a quote Mike mentioned during his talk.

"We will accomplish what we do together. We share our successes & we never let anyone of us fail alone."
- USC Covenant
 

 

This rounds up our 2 day adventure at Techorama, first of all we want to thank everybody for reading our two blog posts and off course a big thank you to the organization of Techorama for creating such an amazing event!! 

Thanks for reading,

Maxim, Sam & Tom

Posted on Tuesday, May 12, 2015 3:44 PM

Maxim Braekman by Maxim Braekman

Sam Neirinck by Sam Neirinck

Tom Kerkhove by Tom Kerkhove

The second edition of Techorama, which is being hosted at Utopolis Mechelen, provided a large range of interesting sessions covering all kind of topics. Read more about some of the sessions from the first day in this post.

The second edition of Techorama again promises to be an interesting event, grouping experts in all kind of technologies to share their knowledge and experiences. Split over two days, there are over 70 sessions. A short summary on some of the sessions of this first day can be found below.

Keynote by Hadi Hariri

The honor of kicking off the entire event went to Hadi Hariri, who got to give an inspiring presentation about the constant chase of developers and tech companies for the mythical "silver bullet". In other words, developers keep looking for the ultimate framework that allows them to build any kind of great application. Because of this constant chase, new frameworks keep popping up and people have been moving from framework to framework, only to discover that this brand-new technology is not perfect as well. Since every framework will have its limitations, the goal to find this silver bullet, remains unreachable.

But with any type of project, the most important idea to keep in mind is to think before you act. Don't just start developing your apps using the newest technology, but consider what would be the best choice for your specific situation.

Keeping in mind that several types of frameworks, technologies and tools are to be subject of several sessions, this keynote started the event off in a very fitting way.

SELECT VALUE FROM DATASTREAM by Alan Smith

For those who were present at the first edition of Techorama, you'll notice a first familiar face. Alan Smith is back, this time giving an insight in the usage of Azure Streaming Analytics by collecting the telemetry data from…, yes, the racing game, he loves using at any demo :)

By sending all of the telemetry data to an event hub, Alan was able to process this data through Streaming analytics to get the average speed, gear, best lap time,… but also to figure out if anyone is cheating. Streaming analytics makes it possible to query the data in any sort of way, allowing you to look for strange/abnormal values, therefore finding cheaters.

As Sam Vanhoutte already gave an extensive description of streaming analytics in this blog post, I will not be diving into this subject, but the demo given by Alan made sure that all of the possibilities were very well illustrated. 

In overall, yet again an interesting and entertaining presentation.

Messaging patterns by Mike Wood

After his talk last year I was looking forward to see Mike Wood in action again! He gave a good session on using a messaging approach in your project and what problems it can fix for you but also what the downsides are.

During the session Mike walked us through some of the concept & patterns used in messaging.
Here are some examples of those discussed :

  • Handle your Poison messages, you don't want to waste resources by trying to process these messages and block everything. It's a best practice to send them to a seperate queue, i.e. a dead-letter queue, so you can keep on processing the other messages.
  • Support Test messages when possible. This allows you to test a specific behavior on a production system while not changing the live-data.
  • Trace your messages so you can visualize the flow of a message. This can help you determine what happens to your messages and where the culprit is when it was lost.
  • Don't lose your messages! By tracing your messages you can follow the trail which a message follows. When using Service Bus Topics, it's possible that the topic swallows your message if there is no matching subscription. One option to handle this is to create a catch-all-subscription.
  • Provide functional transparency in a way that you know what the average processing time is for a specific action so you can pin-point issues and provide alerting on this.
  • Use idempotent processing or provide decompensating logic as an alternative. If your processing is not idempotent you should provide an alternative flow that allows you to rollback the state.

In general, messaging can help you improve the scalability & flexibility by decoupling your solution but this comes with the downside that your complexity increases. Also processing in sequence or using ordering is not easy.

Although I have some experience with messaging it was still a nice session where he give some additional tips on how you can trace your messages better or pinpointing the issues by using the average processing refrences.

Great speaker & great content!

Docker and why it is relevant for developers by Rainer Stropek

One of the benefits of going to a conference is to learn about technologies you would otherwise not pickup easily. It’s also an opportunity to learn about speakers unknown to you. Since I didn’t know Rainer and only had a very (very) high-level knowledge of Docker, this seemed like a good session.

Docker is a platform to facilitate building, shipping and running your application, anywhere. It uses a concept called Container virtualization. This is a level above virtual machines, the container reuses the host operating system. It has the benefit that deployment using Docker is much faster than spinning up a new VM (it can be as little as a second).

What you deploy with Docker is a Docker image. An image is not a monolithic entitity. You can build upon existing images (which can be found on Docker Hub), and only the modifications you do are in your image, the baseimage is referenced in the dockerfile.

Once you setup your Docker image, you can easily deploy it to another environment and be sure it’s setup identically to your Development machine.

All of this and more was covered in Rainer’s session. At the end an ASP.NET 5 application was deployed with Docker, on an Ubuntu machine.

What about Windows one might ask? Docker uses Linux-specific kernel features, which means you’d need to use a lightweight VM to run Docker in a Linux virtual machine. 
However, with the recent announcements of Windows Server Containers and Hyper-V Containers, I think it’ll be very interesting how Microsoft incorporates the container model in both their cloud and on-prem solutions.

The slides of this excellent talk can already be found on his blog.

 

 

That was it for day one, stay tuned for more Techorama action tomorrow!!

 

Thanks for reading,

 

Tom, Sam & Maxim

Posted on Wednesday, April 29, 2015 10:00 PM

Tom Kerkhove by Tom Kerkhove

In this brief blog post I will summarize the extended features to Azure SQL Databases and walk through the new data offerings and give you some pointers for deeper insights.

Today Microsoft announced additional features for Azure SQL Database and two new big data services called Azure Data Lake & Azure SQL Data Warehouse at //BUILD/.

Extending the Azure SQL Database capabilities

Scott Guthrie announced new capabilites for Azure SQL Database going from full text search to creating elastic database pools to encrypting connections with Transparent Data Encryption (TDE) which presumeably uses Azure Key Vault behind the scenes.

Learn more about the extended capabilities here :

  • Build 2015 : Elastic Database Tools (Video)
  • Microsoft Announces Elastic SQL Database Pools For Azure (Article)

SQL Data Warehouse

SQL Data Warehouse allows you to store petabytes of relational data in one place and integrates Machine Learning & Power BI.

While Azure SQL Data Warehouse is released two years after AWS' Redshift Scott compared both services and showed that SQL Data Warehouse now is a leap ahead offering the solution on-prem, is more flexible and comes with full SQL support.

Azure SQL Data Warehouse will become available as public preview in June.

Azure Data Lake

Last but not least is Azure Data Lake, which is in private preview, that allows you to store & manage infinite amount of data and keep it in their original format. This allows you to store your valuable data for ages without losing important segments of it.

Data Lake will be a central storage to perform low latency analytics jobs with enterprise-grade security and integration with other services like Azure Stream Analytics. It is compatible with Hadoops HDFS and Microsofts HDInsights as well as the open-source tooling like Spark & Storm.

Learn more about Azure SQL Data Warehouse & Azure Data Lake here :

  • Microsoft announces Azure SQL Data Warehouse and Azure Data Lake in preview (Article)

  • Microsoft BUILDs its cloud Big Data story (Article)

  • Introduction to the Data Lake-concept (Article)

Current IoT offering in Microsoft Azure

Let's wrap up the day with an nice overview of the current Azure offering in the IoT space.

Let's hope that tomorrows keynote will unveil what the famous Azure IoT Suite includes and what tricks Microsoft has up their sleeves.
In the meanwhile have a look at the IoT breakout sessions today :

  • Internet of Things overview (link)
  • Azure IoT Security (link)
  • Best practices for creating IoT solutions with Azure (link)

All images in this blog posts are property of The Verge & Venturebeat.

Thanks for reading,

Tom.

Categories: Azure IoT
written by: Tom Kerkhove

Posted on Friday, April 17, 2015 5:21 PM

Maxim Braekman by Maxim Braekman

On April 3rd, we had the honor of taking part in the world premier of the IoT Dev Camp organized by Microsoft at their offices in Zaventem, Belgium. Our host of the day was Jan Tielens (@jantielens), who guided us through demo's and labs while using both cloud services and electronics.

In general, it might sound easy to implement a range of devices into a proper interface, but there are a lot of things which have to be taken into account when setting this up. Some of the things you need to keep in mind are device registration, security and keeping connectivity-settings up-to-date.

One of the possibilities to take care of securing the communication between the devices and a cloud service, but preventing you to configure the security every single time, is by using a device-gateway.
This gateway will take care of all the communication and corresponding security, between the devices and cloud service. This allows you to easily add new devices to the interface without adapting the current interface.

 

The goal of this session was to create a solution in which sensors are registering data which is sent and processed by cloud services. Before we could actually start tinkering with devices and sensors ourselves, we got a nice presentation, including some demo's, on how to configure and use Azure App Services, such as Event hubs, streaming analytics and mobile services.

Event hubs

An ideal service to be used for collecting all data coming from several devices are Event Hubs. This enables the collection of event streams at high throughput, from a diverse set of devices and services. As this is a pub-sub ingestion service, this can be used to pass on the data to several other services for further processing or analysis. 

Streaming analytics

Thanks to streaming analytics the data retrieved from the sensors can be analyzed at real-time, showing the most recent, up-to-date information on, for example, a dashboard.

As Sam Vanhoutte already gave an extensive description of streaming analytics in this blog post, I will not be diving into this subject. 

Mobile services

Using Azure Mobile services, you can quickly create a service which can be used to process and show any type of data on a website, mobile device or any other application you could be creating.

This session did not go into the details of creating mobile services with custom methods. This was only used as an example to show that the backend database can be used to store the output when using streaming analytics. In a real-life solution this would allow you to make all of the data, collected from several sensors, publicly available.

Devices

There are several types of devices which can be used as a base to start setting up an IoT-interface. Some of those boards are described in short underneath.

Arduino

The Arduino, which is an open-source device, is probably the most-popular one currently available. The biggest benefit of this device is the large community, which allows you to easily get the required information or samples to get you going.

The down-side of this device, are the low specs. With only 32K of flash memory, it has a limited amount of capabilities. Security-wise, for instance, it is not possible to communicate with services, using the HTTPS-protocol, however it is capable of sending data over HTTP, using an UTP-shield.

More info can be found here: http://arduino.cc/

Netduino

Another device, which is quite similar to the Arduino, is the Netduino, as it is based on the platform of the former. This board has better specs than the Arduino, but is, because of these specs, more power-consuming.

The big difference, however is that it allows you to run the .Net Micro Framework, enabling you to develop using .Net languages.

Then again, the downside of this board is that the community is not as big, meaning you will have to figure out more of the details yourselves.

More info can be found here: http://www.netduino.com/

.Net Gadgeteer

One of the other available development-boards is the Microsoft .Net Gadgeteer, which also enables you to use the .Net Micro Framework and allows you to use a range of "plug-and-play" sensors.

The specs of this device are better than both of the previous boards, meaning it has a lot more capabilities, but then again it does not have a large community helping you out.

More info can be found here: http://www.netmf.com/gadgeteer/

Raspberry Pi

Of all of these boards, the Raspberry Pi is the one with the highest specs, even allowing you to run an operating system on top of the device. Typically, this device is running a version of Linux, but as was announced some time ago, Microsoft will publish a free version of Windows 10 that will be able to run on top of the board!

The huge benefit of this board is the capability of using pretty much any programming language for development, since any required framework can be installed.

However, before you can start using the Raspberry Pi, you will need to obtain a copy of any OS, which has to be 'burned' onto a micro-SD card, that will be acting as the 'hard'-drive of this board.

More info can be found here: http://www.raspberrypi.org/

Closure

Once the presentation and demos were finished, we all got the chance, during a ‘hackathon’, to attempt to set up a fully working flow, starting from sensors and ending up in a cloud service.

On overall, this session gave us a nice overview of the capabilities of IoT in real-life situations.

To round up the session, Microsoft came up with a nice surprise. As this was the world premiere, all of the attendees received a Raspberry Pi 2, preparing us for integrating IoT using Windows 10.

Thank you, Microsoft!

 

 

 

Categories: IoT Azure
written by: Maxim Braekman