Overview

Bloomberg is a file-based connector, meaning that Merge1 is receiving the data in the form of files rather than requesting it from a specific API. In the case of Bloomberg, the connector obtains the data in the form of file sets. A typical file Bloomberg file set looks like the following:

[id].msg.[date].xml -  this file contains the data for Bloomberg emails;
[id].msg.att.[date].tar.gz - This is the archive containing the attachments of the aforementioned emails;
[id].dscl.[date].xml - this contains the disclaimers that are put in those emails;
[id].ib19.[date].xml - this contains the messages sent in Instant Bloomberg, which is their IM platform;
[id].ib19.att.[date].xml - this contains the attachments of the IB messages.

The aforementioned [id] piece is the customer ID given by Bloomberg and it usually has one letter at the front with 4-6 numbers following it. As for the date, it represents the day the data has been generated, and its format is YYMMDD. Additionally, Bloomberg might split these files into multiple pieces if they exceed 1 GB in size.

In this case, the customer will receive multiple XML files with a number added to the extension (such as [id].msg.[date].xml, [id].msg.[date].xml1, [id].msg.[date].xml2, etc.) and multiple Tar.gz archives with the number added before the extension (for example, [id].msg.att.[date].0.tar.gz, [id].msg.att.[date].1.tar.gz, [id].msg.att.[date].2.tar.gz, etc.) depending on the total size of the data.

In that situation, the customer is advised to upgrade the application to the latest version if they are using a build older than 7.0.2304, which is the version that is adapted to the splitting logic of the files.

Please note that these files are usually downloaded in an encrypted form and thus will have *.gpg extension before processing.

History tracking

Bloomberg-generated data is delivered on the next day of each business day (the platform is not working on weekends). Thus, they export data 5 times a week from Tuesday to Saturday. On Sunday and Monday, the connector can be run as a backup in case the production was paused during the week.

After getting uploaded, the files stay at the root of the SFTP site for 21 days, after which they are taken out and need to be requested from Bloomberg to be processed if need be. To make sure that the connector doesn’t bring duplicated data due to an incorrectly working history tracking mechanism, the history tracking parameter  Maintain history of downloaded file for <..> days must be set to 0 (screenshot below), which stands for infinite:

Attachments and disclaimers

A frequently occurring issue with Bloomberg is that upon the initial export of the files, there are often a few attachments and disclaimers missing from the set (yet are referenced within the messages and should’ve been there). In those circumstances, they usually reupload the files after a couple of days with the missing content included. Due to this, there is a strong and complex mechanism developed for the connector to prevent the loss of those files, and this section will describe the best options to use in production.

• The Attachment Validation: must be set to Fail messages with missing attachments.
• Disclaimer Validation: must be set to Fail messages with missing disclaimers.

The settings in the above screenshot are there not to allow file sets to be parsed if there are missing attachments/disclaimers.

Alongside the above, there are a few other features to be enabled:

• Advanced reprocessing

• Full attachment validation
• Full disclaimer validation

Advanced Reprocessing is for reprocessing the source files that had missing attachments or disclaimers, and thus should be used in conjunction with Full Attachment Validation and Full Disclaimer Validation.

When all 3 are enabled, the app processes the source files, and upon detecting a missing attachment/disclaimer it moves them to /[Quarantine_folder]/_ToReprocess directory. During the next session, it attempts to download the files present in the directory again, and upon validating that all content is intact, constructs and sends the messages to the target, thus ensuring that none of the message contents gets lost before the delivery to the archive.

The rest of the settings are optional and thus can be configured according to the business requirements and preferences of the compliance department.