Or: How to get all fields from different sources into a single cache record while ensuring real-time fields are always up to date for snapshots and entitled correctly.
FactSet provides a large set of WebAPIs, SFTP downloads but also real-time APIs for accessing quality market and associated data as well as historical data, reports and much more.
Today customers do not want to deal with the ever-increasing number of APIs and streaming data delivery technologies. Instead, a simple, single point of access for market data, without having to care what kind of technology delivers the data, is in demand.
To comply with this requirement the complete set of data available for a symbol needs to be prepared and made available over high-speed delivery mechanisms ideally in the cloud i.e., a Multi-Source Integrating Cache Array: MuSICA.
The ONE MuSICA (Multi Source Integrating Cache Array)
is the single point of access for all most accurately updated data belonging to one symbol (master key)
provides access either on-prem or out of the cloud or both
delivers a single API for request-response and real-time streaming data
is virtually endlessly scalable by increasing the amount of caching microservices
supports thousands of simultaneous cache sources
lightning-fast cache updating and record delivery (< 0.01 ms latency for > 1Mio requests)
500k+ cache responses/second with 250+ fields
10k+ simultaneous consumers
fully permissioned cache filling and data delivery (entitlement enforcement)
cache arrays are fully resilient and failover time is <1s
all connectivity and low latency data transfer over Solace middleware (VMR)
For a specific customer use case, the following FactSet data from different sources needed to be consolidated into one record:
FactSet broadcast API for real-time streaming market data (incl. OPRA)
Meta data (e.g., Master Security Data), provided via cron every morning at a specific point in time
Reference data via WEB-API, requested by cache fillers running early in the morning
Different WebAPI sources for data not available over broadcast API
Real-time calculated indicators
Result:
No customer side installation required (AWS/Docker); customer only provides connectivity to the cloud VPCs
The MuSICA cache from this use case consolidates five different sources that would have required four different access technologies
data delivery in lightning speed, fully entitled
very simple adaptation of existing applications by using the ONE API for any data (Java/C/C++/.Net)
entitlement for multi-user services using openMECS
All MuSICA data is also available in Excel using snapshots and/or real-time subscriptions.
Author: Mauricio I. González Evans
Contact: mge@bccgi.com