In the recent edition of The BizTalker Newsletter I featured the below article. Read the article below or join the newsletter to have it delivered to your email. I welcome and comments. feedback, or thoughts on this subject.
Server AppFabric contains two core pieces – Hosting and Caching. Today, I’m going to focus on the Caching piece of AppFabric.
AppFabric Caching is a new offering from Microsoft as part of the Server AppFabric collection of technologies for the Application Server role inside Windows. AppFabric Caching is Microsoft’s Distributed Cache solution for on-premise, fast, in-memory, and durable object caching.
A lot of community bloggers have been blogging about the use of AppFabric Caching with BizTalk Server. A few of the best posts are:
- Using Windows Server AppFabric Caching for Storing SSO & Configuration data by Abdul Rafay – This shows how to Cache SSO data inside BizTalk.
- Use AppFabric Cache to cache your WCF Service response by Mikael Hakansson – This shows how to Cache WCF response messages inside BizTalk.
- Using AppFabric Cache in BizTalk by Todd Dhaeyer – This shows how to use AppFabric to Cache reference data and even prevent double messages.
Is Server AppFabric Caching really solving problems or creating new ones?
Rather than diving too deep into AppFabric Caching, let me summarize with a list of Do’s and Don’ts.
AppFabric Rules of Thumb – Don’ts:
-
Never put anything in the Cache that MUST not be lost – it is a non-transactional cache, not a database.
-
Never install AppFabric Caching along with other server products. It will take up all of the available resources of the server including the entire available RAM so it should only be installed on a dedicated server.
-
Avoid making a connection to the Cache Cluster with each request. The initial connection to the Cache Cluster is more expensive than reading and writing to the Cache.
AppFabric Rules of Thumb – Do’s:
-
Use the Cache for fast, in-memory access to serializable .Net Objects that may or may not actually be in the Cache.
-
To achieve fault tolerances in the Cache, simply add a second node to the Cache Cluster (in most cases).
-
Use the Cache to reduce the load, performance, and potentially the license costs of more expensive products like SQL Server.
-
As demand grows, add more servers to the Cache Cluster to increase the available RAM for Caching.
The guidelines seem simple enough, so what is my hold out on BizTalk and AppFabric Cache?
Using BizTalk and AppFabric together makes awesome demos and can simplify some problems like working with reference data. The current release of AppFabric Caching is designed to run alone on its own server since it could take up all available RAM. This would not be a good situation on a production BizTalk Server.
My main concern with AppFabric Caching is if you have a separate caching layer along with your BizTalk Server, why not just use SQL rather than the Cache?
-
Lower Cost – AppFabric Cache is included with the Windows License but you need extra hardware to run it on its own.
-
Better Performance – AppFabric Caching would reduce the load on SQL but I would like to see better data on SQL connection vs. Cache connection costs. Although, if the existing SQL Cluster was already reaching capacity, moving more items away from that would be a plus.
-
Lower Latency – In my mind, reducing latency is the biggest factor for using a Cache. Cache would reduce the latency vs. a call to the database. I would say in most BizTalk Solutions an external call to a SQL Server would probably not be the bottle neck given other BizTalk tuning parameters.
-
Mission Critical Data – SQL wins out here (at least for now) because it has a true transactional backend data store.
I love all three of the AppFabric community demos shown above. At this stage, I would be comfortable using AppFabric Cache for reference and SSO data – not so much for removing double messages unless I was OK with maybe missing one now and then (but then what’s the point?). I would still look to leverage SQL for double message removal given the need for true transactions. Like everything – your unique situation may vary.
In conclusion, BizTalk and AppFabric Caching can work together. Will caching work in scenarios such as a message re-play and de-duper? Absolutely! But for me, given the criticality of most BizTalk installations, I would use caution on using AppFabric Caching with BizTalk at least for now.
Thanks for featuring my article Stephen. Its a good summaryfor now related for using AF caching with BizTalk. Any other scenarios related to BizTalk you can come up with? I agree on the point we cannot use cache for responses and duplicate message checking as cache is not persistent and reliable.
The funny thing was the article I was originally writing was about how great it is to use BizTalk with AppFabric Caching. I changed my position once I found out a little more about system requirements and durability of AppFabric Caching.
I think any of the before mentioned uses of AppFabric are valid as long as you understand the risks. I mean, using it for de-duping works great if you only get a few messages a minute through a single server with no chance of overrunning the RAM.
The scenario I like best is inside the mapper for reference data. Good use of the cache. Could probably use this inside the Orchestration for something similar. But you said it well %u2013 not persistent and reliable. Two words that just don%u2019t mix well with BizTalk.
Great post Stephen! Beats the hell out of Top Model 15 😉
Thanks for the kubos.
Thanks for the very nice Post.
Forgive me if my queries are silly.As a newbie to AppFabric , how do we do updates on the cache & how does it get reflected in Biztalk ?
Also, apart from the caching feature, is there any relationship between using BizTalk & AppFabric together ?
Though both have monitoring,persistence , they differ greatly in Architecture & purpose.
Are we seeing any future of having AppFabric & BizTalk working together .The question are purely on a learning curve point of view.Thanks .
You would need to have custom .Net code to read and write to the AppFabric Cache. So this would need to be called in the pipeline or orchestration. You can see the code for this on some of the samples referenced at the top of the post. It%u2019s rather straight forward once you see the code.
Server AppFabric (not to be confused with Platform Azure AppFabric) has two core pieces: Hosting and Caching. This article focuses on the caching piece.
The hosting piece actually fits together with BizTalk much better. Through both Windows Workflow with AppFabric Connect (http://blogs.msdn.com/b/biztalk_server_team_blog/archive/2010/06/10/biztalk-appfabric-an-introduction.aspx) or simply just hosting WCF Endpoints to use AppFabric%u2019s Auto Start feature (http://blogs.msdn.com/b/paolos/archive/2010/10/30/how-using-appfabric-auto-start-feature-to-avoid-warm-up-delays-for-iis-hosted-wcf-receive-locations.aspx).
I strongly recommend investing time into leaning AppFabric now. One of the best resources is the free Microsoft Jump Start (http://www.microsoft.com/downloads/en/details.aspx?FamilyID=7290f7ed-e86b-4114-a452-4f07fa32403d). It%u2019s a total of 4 days of sessions but well worth it!!!
As I see it, BizTalk and all forms of AppFabric (Cloud and Server) are here to stay. The faster you can learn how these technologies work together (and even AppFabric independently) the better you will be down the road.
Hope this helps.
Hi Stephen
Million Thanks for all the answers .
Helped me to choose a path & direction.
Thanks Again.
Paolo Salvatori and I, had this idea about using AppFabric Cache as a workflow persistence provider.
Paolo Salvatori and I, had this idea about using AppFabric Cache as a workflow persistence provider.
Thanks for the article and the links.
Regarding the point on the connection:
“Avoid making a connection to the Cache Cluster with each request.%u00a0 The initial connection to the Cache Cluster is more expensive than reading and writing to the Cache.%u00a0”
What would be the best way to get around this in a BizTalk solution? One wouldn’d want every pipeline instance spawned, or every map (calling a custom assembly to lookup some reference data) to try and establish a connection to the cache, right?
One almost needs to cache the connection itself somewhere so that biztalk components can reuse the connection and not have establish connections themselves.
Any thoughts?