Orchestration is the ability to control operational flows and activities based on business rules, especially in multi-application systems complicated enough to require middleware such as ESB (“Enterprise Service Bus”) or the older MOM (“Message-Oriented Middleware”).
In the context of a file transfer system, orchestration often refers to the ability to apply automation such as triggers, schedules, explicit calls and chained calls to model or solve a business problem.
In the context of SOA (“Service Oriented Architecture”), orchestration typically refers to the ability of programmers to rapidly develop composite applications due to the fact that most available application APIs have been encapsulated and published in reliable directories that programmers’ applications can easily interpret and use.
BEST PRACTICE: Orchestration typically invokes an image of “drag and drop” business application development: a task easy enough for the average business analyst. Reality often requires more than that: shelling out to scripts, editing raw XML documents by hand and having to clean up “orchestrated code” after an incompatible interface is rolled out are still common issues.
Middleware is a software architecture concept that refers to integration of disparate applications to facilitate reliable communication. Middleware frequently relies on encapsulating inter-application communications in the concept of an “message”, and often has the ability to queue or perform optimized delivery or copying of messages to various applications.
Common types of middleware include EAI (“Enterprise Application Integration”) middleware such as ESB (“Enterprise Service Bus”) or the older MOM (“Message-Oriented Middleware”).
File transfer applications are themselves often used as middleware, helping to facilitate bulk data transfers between applications using standards such as FTP. Managed file transfers often include the ability to perform some intelligent routing of data and sensitivity to particular transmission windows set by the business.
ESB is short for “Enterprise Service Bus“, a modern integration technology used to quickly tie heterogeneous applications across different operating systems, platforms and deployment models.
Enterprise Application Integration (“EAI”) is a methodology which balances seamless experience across heterogeneous enterprise applications and datasets of various origins, scope and capability with the need to make major changes to those applications or datasets.
Today, EAI often uses ESB (“Enterprise Service Bus”) infrastructure to allow these various applications to communicate with each other. Before ESB, MOM (“Message-Oriented Middleware”) would have been used instead.
Today’s convergence of file transfer and EAI systems was foretold by Steve Cragg’s 2003 white paper entitled “File Transfer for the Future – Using modern file transfer solutions as part of an EAI strategy”. In that paper, Cragg wrote that, “judicious use of file transfer in its modern form as part of an overall EAI strategy can reduce overall business risk, deliver an attractive level of ROI, speed time to market for new services and enable new business opportunities quickly (for example B2B).”
An Enterprise Service Bus (“ESB”) is a modern integration concept that refers to architectural patterns or specific technologies designed to rapidly interconnect heterogeneous applications across different operating systems, platforms and deployment models.
ESBs include a set of capabilities that speed and standardize a Service-Oriented Architecture (“SOA”), including service creation and mediation, routing, data transformation, and management of messages between endpoints.
With the rise of SOA in the mid-2000’s, ESBs took over from MOM (“Message-Oriented Middleware”) as the leading technology behind EAI (“Enterprise Application Integration”).
Examples of commonly deployed ESBs include MuleSoft’s open source Mule ESB, IBM WebSphere ESB, Red Hat JBoss and Oracle ESB. The Java Business Integration project (“JBI”) from Apache is also often referred to as an ESB.
EAI is short for “Enterprise Application Integration“, a methodology which balances seamless experience across heterogeneous enterprise applications and datasets of various origins, scope and capability with the need to make major changes to those applications or datasets.
Message-Oriented Middleware (“MOM”) is software that delivers robust messaging capabilities across heterogeneous operation systems and application environments. Up through the early 2000’s MOM was the backbone of most EAI (“Enterprise Application Integration”) inter-application connectivity. Today, that role largely belongs to to ESB (“Enterprise Service Bus”) infrastructure instead.
In the context of file transfer, MOM stands for “Message-Oriented Middleware“, which is software that delivers robust messaging capabilities across heterogeneous operation systems and application environments.
The term “package” can mean different things in different file transfer situations.
“Installation package” – A file that contains all the executables, installation scripts and other data needed to install a particular application. This file is usually a compressed file and is often a self-extracting compressed file.
“Package sent to another person” – Very similar in scope to email’s “message with attachments”. This is a term that has rapidly gained acceptance (since about 2008) to describe what gets sent in “person-to-person” transmission. A package may contain zero or more files and a plain or richly formatted text message as its payload. A package will also always contain system-specific metadata such as sender/receiver identity, access control attributes, time-to-live information and quality of service information.
“Installation package” is the earlier context of the term “package” ; if you’re dealing with server teams or transmission specialists who deal primarily with system-to-system transfers then “installation package” is probably what they mean when they say “package”.
“Package sent to another person” has evolved as file transfer vendors gradually align the terminology of person-to-person file transfers with physical parcel transfers like those done by UPS or FedEx. In physical parcel transfers, individual packages may contain a variety of things but each is specifically addressed, guaranteed to be delivered safely and intact and each has its own level of service (e.g., 2nd day vs. overnight). The term “packages” is similarly used with many person-to-person file transfer solutions to help non-technical people understand the concept in a different context.
“Community Management” is a marketing term used to describe technology and services that use external authentication technology to provision (or “onboard“) users or partners using rich profile definitions and which allows users and partners to maintain elements of their own profiles (e.g., contacts, email addresses, member users with limited rights, etc.).
File transfer and/or EDI solutions that provide community management capabilities are either VANs or direct competitors to VANs.
VAN stands for “Value Added Network”. A VAN is a data transfer service that uses EDI and/or file transfer protocols to connect to dozens, hundreds or even thousands of businesses. VANs are often industry-specific; the ones that are will usually connect to almost every major supplier and consumer within that industry (e.g., auto parts).
As a marketing term, VAN is associated with a bygone era. Most VAN solutions now position themselves as providers of cloud-based EDI and file transfer services (e.g., GXS) or EDI and file transfer technology that completely integrates with modern Community Management services (e.g., IBM’S Sterling Commerce).
However, as a technology concept, VANs are what many “cloud” services hope to be when they grow up: a mature and reliable interconnection of most of the major businesses that serve an industry.
BEST PRACTICE: VANs represent the final and most complete evolution of outsourced technology and services in the file transfer industry. However, common VAN pricing practices such as “per byte charges” allowed innovative firms like Standard Networks and Tumbleweed Communications to create a new “managed file transfer” space in the early 2000’s. Managed file transfer technology, even deployed on premises, often provides a significant cost savings over VANs and should be considered by anyone using or thinking of using a VAN service.
To onboard a user or onboard a partner is to set up all the necessary user accounts, permissions, workflow definitions and other elements necessary to engage in electronic transfers of information with those users and partners.
Automatic onboarding of users or partners usually involves external authentication technology of some kind. When that technology involves particularly rich user or partner profiles and allows users and partners to maintain their own information, then the external authentication technology used to onboard users and partners is often called “Community Management” technology.
“On board” and “on-board” are also occasionally used instead of “onboard”, and administrators often use the phrases “onboard a user” and “provision a user” interchangeably. See “provisioning” for more information.
Cyber liability is the risk posed by conducting business over the Internet, over other networks or using electronic storage technology. Insurance can be bought and “risk based” security strategies can be used to mitigate against both the first- and third-party risks caused by cyber liability.
A “first party” cyber liability occurs when your own information is breached. For example, a hack that results in the exposure of your own trade secrets would create a first party cyber liability.
A “third party” cyber liability occurs when customer or partner information your organization has promised to keep safe is breached. For example, a hack that results in the exposure of your customer’s Social Security numbers would create a third party cyber liability.
Companies have compelling reasons to avoid both types of cyber liability, but third party cyber liabilities can be devastating. First party cyber liabilities threaten a company’s competitiveness, but third party cyber liabilities often ruin brands, open the door to million-dollar lawsuits and trigger statutory fines (e.g., HIPAA HITECH’s $50,000 per-incident “willful neglect” fine).
File transfer technology frequently transmits information whose disclosure would lead to first- or third-party cyber liabilities.
BEST PRACTICE: File transfer technology, policy and procedure decisions should be made under the auspices of a risk-based security strategy that takes into account both first- and third-party cyber liabilities. (File Transfer Consulting can help you identify your risk tolerance, then develop a package of file transfer products and services that specifically matches your ideal tolerance.)
B2B (“business to business”) is a market definition (a “space”) that covers technology and services that allow file and other transmissions to be performed between businesses (i.e., not between consumers). B2B covers a lot of conceptual ground, from simple “file transfer” and “secure file transfer” to more sophisticated “managed file transfer” and up through traditional EDI. In addition to overlapping with the venerable EDI space, many analysts now see that the B2B space overlaps with the EAI (“enterprise application integration”) space, the “Community Management” space (e.g., effortless provisioning) and even the hot new cloud services space.
If you are an IT administrator or someone else charged with getting a file transfer system to work, the presence of the “B2B” term in your project description should tell you to expect to use the FTP/S and SSH (SFTP) protocols, although you may also see AS2, proprietary protocols or some use of HTTP/S (especially if web services are available).
If you are a buyer of file transfer technology and services, the presence of the “B2B” term in your project requirements will limit the field of vendors who can win the business, attract the attention of traditional IT analyst firms (e.g., IDC, Forrester and Gartner) hoping to steer the business to their clients and may kick decision making further up your corporate hierarchy. (These effects may be good or bad for your particular project – after having participated in hundreds of file transfer RFPs, the people of File Transfer Consulting may help your project thread the appropriate needle in your organization.)
“FIPS validated” is a label that indicates that the cryptography used in a particular solution implements some or all the algorithms specified in FIPS 140-2 (e.g., AES) and that the underlying cryptography component has been validated by NIST laboratories. See “FIPS compliant” for a weaker statement.
“FIPS compliant” is a slippery phrase that often indicates that the cryptography used in a particular solution implements some or all the algorithms specified in FIPS 140-2 (e.g., AES) but that the underlying cryptography component has not been validated by NIST laboratories.
“FIPS validated” is much stronger statement.