Data File Formats in Middleware, Integration, or EDI software

Data File Formats in Middleware, Integration, or EDI software

Data File Formats in Middleware, Integration, or EDI software

The purpose of Middleware, Integration, or EDI software is to transfer data from one trading partner to another (e.g. transfer Order information from customer to supplier), or from one business application to another (e.g. copy Order information from customer management system to warehouse management system). Many of these transfers are done via data files, and a variety of file formats are available. Some of the common ones are described below.

File Format


“Separated” Files e.g. comma separated,                                          Tab-separated

A common format that uses a specific character (e.g. comma, or Tab) as a separator between each data value, and with one data record per line in the file. Can include an optional first line with data field names. Different record layouts can be used within each file, e.g. one layout for “header” records and another layout for “line” records.

The beginning of a separated file might look like:  “REC_TYPE”,”CUST_CODE”,”ORD_NUM”,”ORD_VALUE”,... “H”,“ABC”,”12345”,100.00,... or like:H<tab>ABC<tab>12345<tab>100.00<tab>...                                                                                  =(where <tab> represents the Tab character)


A format in which each data value has a specific starting position and length within each record (based on number of characters). Usually requires “padding” data values with spaces or zeroes to maintain the correct character positions.

The beginning of a fixed-length file might look like:                                                                                HABC 12345 00000100.00...


“Extensible Markup Language”. Designed as a means of describing and capturing data in a flexible, and potentially unlimited, way by including both the names and values of data elements throughout an XML structure. XML structures can be described in XML “schema” files, so that senders and recipients can work consistently with a set of known layouts (e.g.for Orders, Invoices, etc). XML can be used for individual data files, and often used in conjunction with “SOAP or “REST” web services.

The beginning of an XML file might look like:                                                                                                 <?xml ...>                                                                                                                                    <OrderHeader>                                                                                        <CustomerCode>ABC</CustomerCode>                                        <OrderNumber>12345</OrderNumber>                                            <OrderValue>100.00</OrderValue>



EDIFACT and X12 are both segment-based file formats: the data content is structured into many relatively small, coded segments (e.g. UNH, BGM, NAD). They also share the concept of each file representing an “interchange” between sender and recipient and each interchange can hold more than a single “message” (e.g. multiple Orders and/or Invoices). EDIFACT tends to be used by European organisations, and X12 by those in the USA. They are relatively compact formats that generate small data files, and were an excellent option when networks did not have the transfer speeds that they have today.

The beginning of an EDIFACT file might look like:  UNA:...UNB+...’UNH+001+ORDERS...’BGM+220+12345...’NAD+BY+ABC...


“JavaScript Object Notation”. Similar to XML in the sense that it can contain both the name and value of each data element, but in a more concise form. Commonly used for “REST” web services.

The beginning of a JSON file might look like:                                                                        {“OrderHeader”:{                                                                                                                "CustomerCode":"ABC",                                                                                            "OrderNumber":"12345",                                                                                                       "OrderValue":100.00,

Excel (.xls, .xlsx, etc)

A proprietary file format used to store workbooks (“spreadsheets”) created in Microsoft Excel. The row/column nature of a spreadsheet makes it a useful way to structure and manage business data, and Excel has become common enough that most businesses can work with it.


Adobe’s proprietary “Portable Document Format”. Intended as a means of capturing documents in a way that would allow consistent presentation across various types of printers and computers, and so successful in that role that its use has become very widespread. However, PDF is primarily about presentation to human viewers and is not intended as a data transfer format.

Each of the above non-proprietary file formats is fundamentally “plain text”, i.e. they usually contain only the characters that can be found on a standard keyboard and therefore are easily opened and displayed by text-editing or word-processing software (e.g. Notepad, Word). These formats can be used to transfer data that contains other characters, but usually only by converting – or “encoding” – those characters into a plain-text equivalent (e.g. “Base64” encoding) and then having them decoded by the recipient of the file.


Browse Articles

101 Uses for Flow

Topics: 10


Topics: 10

Flow Software

Topics: 49


Topics: 42


Contact us

Contact us

Contact us about Integration and eCommerce EDI and we will respond ASAP!... Learn more

Flow Integration

Flow Integration

Use Flow as an integration tool to transfer data between your line-of-business applications... Learn more

Flow EDI

Flow EDI

Flow provides you the capability to trade electronically with your trading partners... Learn more