New SFTP adapter in Biztalk 2016

WCF WinSCP-based SFTP Adapter

Since Microsoft now supplies a SFTP Adapter with Biztalk Server 2016, we thought we might as well try it out instead of using the Blogical one (which we have had several problems with).

It is relatively straightforward to use it in static sendports and receivelocations – the only thing we miss, is a possibility to use temporary filenames during transmission, which it does not support – only temporary folders.

However, it does not seem to be documented how to use the adapter in a dynamic sendport (at least, we have been unable to find it)

As documented, the SFTP Adapter uses the WinSCP program to do its actual work and specifically, it requires the WinSCPnet.dll to do its work.

Microsoft have implemented the WCF adapter in this DLL which wraps the calls to WinSCP.

C:\Program Files (x86)\Microsoft BizTalk Server 2016\Microsoft.BizTalk.Adapter.Sftp.dll – Assembly version 3.0.1.0

The Adapter loads the WinSCPnet DLL in the class SftpHelper.

The dynamic properties can be found in the class SftpOutputChannel Method AddPropertiesFromMessageContext.

Context Property	Description
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#ClientAuthenticationMode	this.sendPropertyBag.ClientAuthenticationMode = (ClientAuthenticationMode) Enum.Parse(typeof (ClientAuthenticationMode), property.Value.ToString()); enum Password, PublicKeyAuthentication, MultiFactorAuthentication
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#UserName	Either specify UserName and Password, or specify a keyfile in PrivateKeyFile
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#Password	Password
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#PrivateKeyFile	Full path to Private Key
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#PrivateKeyPassword	Password to the private key, if it is password protected
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#AccessAnyServerHostKey	If False, host fingerprint must be specified. If True, all hosts will be accepted
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#SSHServerHostKeyFingerPrint	Specify the fingerprint of SSH servers HostKey, if “AccessAnyServerHostKey” is False
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#AppendIfExists	If False, transfer will fail if a file with the filename already exists. If True, the contents will be appended to the existing file We have not been able to make the “AppendIfExists” work. If this property is true, the adapter issues a get command for the filename, and if the file does not exist yet (first transfer), then the transmission fails.
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#TargetFileName	The name of the file created on the SFTP server. You can use the macros specified below
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#TemporaryFolder	this.sendPropertyBag.TemporaryFolder = property.Value.ToString();
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#ProxyAddress	Address of a proxy-server, if you need to use one, or blank.
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#ProxyUserName	Username and password for the proxy-server, or blank if they are not needed
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#ProxyPassword
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#ProxyPort	Port for the proxy-server
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#ProxyType	this.sendPropertyBag.ProxyType = (Microsoft.BizTalk.Adapter.Sftp.ProxyType) Enum.Parse(typeof (Microsoft.BizTalk.Adapter.Sftp.ProxyType), property.Value.ToString()); enum None, SOCKS4, SOCKS5, HTTP
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#EncryptionCipher	this.sendPropertyBag.EncryptionCipher = (EncryptionCipher) Enum.Parse(typeof (EncryptionCipher), property.Value.ToString()); enum Auto, AES, Arcfour, BlowFish, TripleDES, DES
http://schemas.microsoft.com/BizTalk/2012/Adapter/sftp-properties#Log	this.sendPropertyBag.Log = property.Value.ToString();
http://schemas.microsoft.com/BizTalk/2003/system-properties#OutboundTransportLocation	Contains the full EndPointAddress of the server – on the form sftp://servername:port this.ParseSftpUrl(property.Value.ToString());

OutboundTransportLocation needs to be ‘sftp’ (case insensitive compare)

Supported macros

Macro	Value
Use fixed context-property
%MessageID%	http://schemas.microsoft.com/BizTalk/2003/system-properties#BizTalkMessageID
%SourceFileName%	http://schemas.microsoft.com/BizTalk/2003/file-properties#ReceivedFileName
Use the macro as the name of a context-property Use the contextproperty with the namespace namespace http://schemas.microsoft.com/BizTalk/2003/system-properties# with the macroname (nix the percent-signs) appended
%DestinationParty%
%DestinationPartyQualifier%
%SourceParty%
%SourcePartyQualifier
These macros formats either DateTime.Now or DateTime.UtcNow in the specified format
%datetime%	yyyy-MM-ddThhmmss
%datetime_bts2000%	yyyyMMddhhmmssf
%datetime.tz%	yyyy-MM-ddThhmmsszzz
%time%	hhmmss
%time.tz%	hhmmsszzz

Biztalk – Orchestrations gather up in status “Ready to run”

In a previous post, I wrote about a situation we encountered on our production Biztalk 2016 system.

After a series of not-Biztalk related issues (including the MSDTC service crashing), we ended up in a situation, where one of our orchestrations would not process any messages. Existing instances would not process, and new instances would only pile up in status “Ready to run”. NONE of them would change to status “Active”, even after restarting the hostinstance bound to that orchestration.

We could not see any hints anywhere (like eventlog on Biztalk- and SQL server, SQL-server locks or similar) as to why the orchestrations would not be processed.

We run a ESB-like architecture, and since that orchestration was part of the main ESB itinerary, no documents could in fact be fully processed.

In the end, we were able to get most the of the orchestration serviceinstances processed, by Suspending all the serviceinstances in status “Ready to Run” and then “Resubmit” them 1 or 2 at a time. If we took more than a few, the situation would lock up again, and we again would have to Suspend all and resubmit them a few at a time.

In the end we had 5 serviceinstances that simply would not process. These 5 were terminated (they were already saved in our Tracking) and after a hostinstance restart (that may not have been necessary), the orchestration would once again process messages without problems.

Another suggestion we had to resolve this situation, was to remove and then recreate all hostinstances for the host which had problems. We did not try this method.

Extra permission needed for Biztalk File Adapter

If you try to pickup files with the standard Biztalk File Adapter, you will receive these two errors in the eventlog (and the Receive Location will disable) – even if you have givet it MODIFY access to the folder.

The Messaging Engine failed to add a receive location "RL_Test_EDIFACT" with URL "C:\FileDrop\Test\EDIFACT\*.edi" to the adapter "FILE". Reason: "File transport does not have read/write privileges for receive location "C:\FileDrop\Test\EDIFACT\". ".

followed by

The receive location "RL_Test_EDIFACT" with URL "C:\FileDrop\Test\EDIFACT\*.edi" is shutting down. Details:"The Messaging Engine failed while notifying an adapter of its configuration. ".

That is because the Biztalk File Adapter expects “Full Control” on the folders it reads from, but if that is not possible, you can also just add a single extra permission

https://social.technet.microsoft.com/wiki/contents/articles/7008.biztalk-file-adapter-permission-issue-workaround.aspx

Apart from MODIFY access, add the “Delete Sub Folders and Files” Advanced permission to enable the Biztalk File Adapter to read files

Required NTFS permissions for BizTalk File Adapter connecting to network share