When you write a custom pipeline component, loading entire message into memory should be avoided. If you use Memory Stream for large files, it loads entire message into memory and you may have OutOfMemory exceptions. On the other hand, VirtualStream uses Biztalk’s buffering directory after a threshold value which is specified in object initialization.

In addition to this, default message stream can be non-seekable and you may have exception when you try to change message data postion. The solution for this to use ReadOnlySeekableStream which wraps MemoryStream by default.

As a result of these, It is a good approach to use ReadOnlySeekableStream and VirtualStream classes exposed by Microsoft.Biztalk.Streaming.dll together in your custom pipeline components.

Please check below code snippet to see how to use ReadOnlySeekableStream vith VirtualStream.

int bufferSize = 0×280;
int thresholdSize = 0×100000;
if (!inmsg.BodyPart.GetOriginalDataStream().CanSeek)
{
    Stream virtualStream = new VirtualStream(bufferSize, thresholdSize);
    ReadOnlySeekableStream seekableStream =
        new ReadOnlySeekableStream(inmsg.BodyPart.GetOriginalDataStream());
    Stream seekStream =
        new ReadOnlySeekableStream(inmsg.BodyPart.GetOriginalDataStream(),
            virtualStream, bufferSize);
    inmsg.BodyPart.Data = seekableStream;
}
inmsg.BodyPart.Data.Position = 0;

Tweet This Post

I have started to use .local dns extension for my local network. After changing my complete network configuration I realized that some of my Ubuntu machines can resolve the FQDNs ending with .local (Ex: testhost.digitalpains.local ) correctly.

ali@advance17:/home/ali$ ping testhost.digitalpains.local
ping: unknown host testhost.digitalpains.local
 
ali@advance17:/home/ali$ ping testhost
PING testhost.feasiblesolutions.local (192.168.7.2) 56(84) bytes of data.
64 bytes from 192.168.7.2: icmp_seq=1 ttl=64 time=1.02 ms
64 bytes from 192.168.7.2: icmp_seq=2 ttl=64 time=1.12 ms

After some research it turned out that avahi-daemon was interfering the DNS resolve process and FQDNs ending with .local extension wasn’t being resolved.

After I stopped the avahi-daemon service I was able to resolve

ali@advance17:/home/ali$ sudo stop avahi-daemon
avahi-daemon stop/waiting
 
ali@advance17:/home/ali$ ping testhost.feasiblesolutions.local
PING testhost.feasiblesolutions.local (192.168.7.2) 56(84) bytes of data.
64 bytes from 192.168.7.2: icmp_seq=1 ttl=64 time=2.05 ms
64 bytes from 192.168.7.2: icmp_seq=2 ttl=64 time=1.16 ms
64 bytes from 192.168.7.2: icmp_seq=3 ttl=64 time=4.25 ms

After reboot avahi-daemon service will start automatically. On Ubuntu Karmic edition adding the following line to you r /etc/avahi/avahi-daemon.conf file will solve the problem permanently.

AVAHI_DAEMON_DETECT_LOCAL=0

Tweet This Post

Tweet This Post

• Formal Apache Maven site
• AvaJava Web Tutorials‘ Maven category. Really useful web site, teaches common maven configurations and tasks with screenshots and short explanations
• Maven: The Definitive Guide published by O’Reilly. It’s free and you can read full content of the book from the website. Both, html and pdf versions are available.

Tweet This Post

As the mobile phones get smarter, they became an integral part of our life and very tempting attack vector. Today I red an article Voice Encryption: 9 out of 10 Products are Worthless (Technical Description) it is a nice example of with a little bit of creativity how easily you can bypass an expensive security measure.
Author is using a product called Flexispay you can see the flash movies explaining how does the product installed and works. With an investment of 100 Euros a year, you are able too wiretap a mobile telephone less than 5 minutes.

Tweet This Post

There are different options for automatically updating your server. At this article we will use the unattended-upgrades package.

We will start with installing the unattended-upgrades package.

kok@fhome:~$ sudo apt-get install unattended-upgrades

After installing the unattended-upgrades package we need to add  the text below to the  /etc/apt/apt.conf.d/10periodic file.

APT::Periodic::Update-Package-Lists “1″;
APT::Periodic::Download-Upgradeable-Packages “1″;
APT::Periodic::AutocleanInterval “5″;
APT::Periodic::Unattended-Upgrade “1″;

After this modification your system will install the security updates automatically. You can see the details of unattended-upgrades from the log files  /var/log/unattended-upgrades/ folder.

For more information about this subject you can check the Ubuntu wiki page.

Tweet This Post

My problem is BizTalk Ftp Adapter’s inability to run in 64-bit mode… Actually, this is one of the problems of the ftp adapter but I won’t write about it in this entry, I can live with it….

• Out-of-the box Ftp Adapter receive location deletes files from the remote ftp site so it requires write permission on the remote ftp directory, but some of our customers don’t want us to delete file after we retrieve it. For example, some of our customers share same file with lots of recipients on their private ftp server and they do not want to give different ftp directories and put the same file for each recipient.
  In my company, we solved the problem by writing a simple custom ftp receive adapter for those customers who don’t give write permission on their private ftp server. Our custom FTP adapter keeps track of received file properties (like name, size, file-mask, etc.) on a custom database and before receiving a new file, it checks the database whether the file received previously or not. In addition to this, after the file is picked up,we compare CRC of the current file with previously received files’ CRCs to prevent duplicate message processing.
  If you don’t want to write your own custom adapter, You can also look for a third party adapter like /n software Ftp Adapter, which has a “never delete” mode while retrieving files.
• We process large files in BizTalk Server 2006 R2. BizTalk retrieves files by using Ftp Receive Location and our customers also put their files to the directory by using ftp. Because of BizTalk Ftp Receive location is not aware of whether file upload is completed or not, sometimes receive location retrieves incomplete files and that cause BizTalk to fail processing those files.
  In my opinion, Ftp protocol’s lack of functionality to check whether file upload is completed or not shouldn’t be an excuse for the ftp adapter to retrieve incomplete files.
  So, I listed below some of my proposed solutions to handle incomplete file processing:
  Solution 1 : We can write a custom FTP adapter. Custom adapter checks file size before retrieving the file and if its size doesn’t change for a certain time (lets’ say 1 minute) then assumes it is uploaded completely and then process otherwise don’t process and try to retrieve file at next polling.
  Solution 2 : If the ftp server is in our control we can make customers to upload documents to the customer specific temp directories and a program which runs on the ftp server can move uploaded files from temp directories to the directories where BizTalk receive location is polling. While moving file from a temp directory on FTP server would let us have a lock on the file and so we would get a 550 error when trying to retrieve the file at this time. Receive location tries to retrieve file at next polling. Actually, I didn’t tried the solution but I expect it to work. Even it works, not my favorite.
  Solution 3 : Find a third party adapter. /n Software Ftp Adapter may be an option but I don’t know whether it provides a solution to check file completeness or not.

Last but the not least, lack of extension point of adapter framework.
We have to use the adapter as it is or we must find another adapter. We have two options; We can develop our own custom adapter or we can find a third party adapter.
I think, it would be nice if we would not write a whole new adapter and if there were some extension points which we could alter the way of adapter’s processing, by writing something like interceptors. We can intercept the processing flow of the adapter, make some validations and decide whether to let adapter to continue processing or make it stop by throwing an exception or make it skip this time and try to retrieve file at next polling.

Tweet This Post

If you are having this error message either your web user doesn’t have write rights to your installation folders or Path to Temp-folder server parameter is pointing to a wrong location. In my case it was the second. After I changed my Path to Temp-folder parameter problem is solved.

You can change this parameter from Global Configuration > Server page.

Tweet This Post

I want to share some of my notes about Biztalk Server 2006 performance in terms of programming and configuration. This is just a startup and I will try to share my future findings about Biztalk performance in this entry. I also put some related links at the end of the entry for detailed information.

• Change the setting for the %temp% environment variable of Biztalk Server computer to a dedicated, non-system disk.By default, documents that are buffered to the file system during mapping are written to the %temp% directory of the BizTalk Server computer.
• Use VirtualStream class especially for large file processing in your pipeline components. If you use MemoryStream for large files you may have OutOfMemory exceptions. Virtual Stream uses Biztalk’s buffering directory after a threshold value which is specified in object initialization.
• Use 64 bit host instances because of memory constraint of 32 bit hosts. But some of Biztalk Adapters(FTP, POP3, SQL) and BiztalkExplorerOM class doesn’t run in 64-bit mode. You can continue to use 32 bit host instances for the items which don’t run in 64-bit mode but be careful not to process large files on 32-bit host instances. Because a 32 bit host instance process may only grow to 3 ~ 4 GB Max, even your server has more physical memory space. Please also check “Memory Limits for Windows Releases” document.
  8 GB of memory should be considered the minimum for 64-bit Windows because generally 64 bit windows require more memory than 32 bit windows.
• One of MSDN document says that ”while processing large files minimize the use of maps in orchestrations”, but it’s a disputed suggestion for me. Because as I mentioned previously, some of Biztalk Adapters (like FTP, POP3 and SQL) does not run in 64-bit mode. Let’s say you are doing a large flat file mapping operation in a pipeline component and it is running on an ftp receive location. You should also consider that your flat file size will be much more when it is disassembled to an xml message and that may cause an out of memory exception. To overcome the memory problem you can use a dynamic mapper orchestration which is running on 64 bit host instance and If you are doing your mapping in a pipeline component, you can also call your receive pipeline from the orchestration.
• Configure antivirus software to avoid real-time scanning of BizTalk Server executables and file drops.
• Minimize use of XmlDocument in .net code and orchestration. All message content loaded into memory while processed with XMDocument and instance’s memory usage becomes around 10 times of the original xml message.
  When passing xml messages from your orchestration to your .net code use XLANGMessage instead of XmlDocument. You should also use stream approach especially for large files.

// Stream approach
public static void processXLANGMessage( XLANGMessage msg )
{
   StreamReader reader = new StreamReader(msg[0].RetriveAs(typeof(Stream));
}

• Biztalk 2006 Performance Handbook
• How BizTalk Server Processes Large Messages
• Orchestration Performance – From Jon Flanders’ Blog
• Optimizing Operating System Performance
• Finding and Eliminating Bottlenecks
• Optimizing Performance for BizTalk Server
• Troubleshooting BizTalk Server Performance

Tweet This Post

Most of the  production servers  has limited connectivity to the Internet. If they have any connectivity then probably it  will be through a forward proxy server.

If your server is behind the proxy server, then you need to provide proxy server details in /etc/yum.conf.. The proxy setting must specify the proxy server as a complete URL, including the TCP port number. If your proxy server requires a username and password, specify these by adding proxy_username and proxy_password settings.

The settings below enable yum to use the proxy server 192.168.5.100, connecting to port 3128, with the username yum-user and the password qwerty.

# The proxy server - proxy server:port number
proxy=http:/192.168.5.100:3128
# The account details for yum connections
proxy_username=yum-user
proxy_password=qwerty

Tweet This Post