June 2, 2026
Infor Streaming Pipelines cap at around 165,000 transactions per day. Here's why initial loads take so long and how to...
Infor data extraction has three real paths: Streaming Pipelines, the Data Lake API, or Mirrordex . Each one gets your CloudSuite data into SQL Server, Snowflake, or anywhere else, and each has limits the documentation doesn’t make obvious. Streaming Pipelines caps at around 165,000 transactions per day. The Data Lake API caps each result at 100,000 rows or 10MB and limits your total daily compute time. Mirrordex streams from the transaction log with no equivalent limits but requires database access.
Here’s what each option actually does, where each breaks, and how to pick the right one.
Streaming Pipelines is Infor’s vendor-managed service for delivering CloudSuite data to external systems. You configure pipelines in the Infor admin console, point them at Azure SQL, and Infor handles the data movement.
The Data Lake API lets you query Infor’s Compass-based Data Lake directly and pull data as CSV. You write a small app that authenticates, calls the Compass v2 API, downloads results, and loads them into your destination.
Pulling data isn’t a single call. The Compass v2 API requires three steps:
var csv = await dataLakeClient.GetTableAsCsv("oeeh", lastExtractedDateTime);
using var reader = new CsvDataReader(csv);
using var bulk = new SqlBulkCopy(connectionString) {
DestinationTableName = "dbo.oeeh",
BatchSize = 10000
};
await bulk.WriteToServerAsync(reader);
Important: SqlBulkCopy is insert-only. It moves rows fast, but it doesn’t update existing rows or delete removed ones. For initial loads, this is fine. For ongoing replication, you need additional logic to handle updates and deletes, typically a merge step or change tracking on top of the bulk load.
DL Sync is Cesova’s log-based replication tool for Infor CSD and ISM. It runs in your environment, reads transaction logs from the Infor source, and replicates every insert, update, and delete to SQL Server (on-premise or cloud) in near real time. No transaction throttle, no API page limits, no daily compute cap.
Mirrordex reads the database transaction log directly rather than querying tables or calling APIs. Every committed change in Infor (insert, update, or delete) flows through to the destination SQL Server within seconds. Replication is continuous, not scheduled. The destination schema and tables are created and kept in sync automatically, with native Infor datatypes preserved.
|
Key Parameters
|
Streaming Pipelines
|
Data Lake API
|
Mirrordex
|
|---|---|---|---|
|
Latency
|
Near real-time
|
Polling interval
|
Near real-time
|
|
Source load
|
Low
|
Medium
|
Negligible
|
|
Throttle/limits
|
~165K txn/day
|
100K rows or 10MB per result; daily compute cap
|
None
|
|
Captures
|
Inserts + updates
|
Whatever you query (bulk copy = inserts only)
|
Inserts + updates + deletes
|
|
Destination
|
Azure SQL only
|
Anywhere you load it
|
SQL Server (on-prem or cloud)
|
|
Schema
|
Generic datatypes
|
You define
|
Native Infor types preserved
|
|
Setup effort
|
Low
|
Medium-high
|
Low (Cesova installs)
|
|
Multi-tenant SaaS
|
Yes
|
Yes
|
Not currently
|
|
CSD / ISM (on-prem)
|
Yes
|
Yes
|
Yes
|
Most Streaming Pipelines contracts cap each pipeline at around 165,000 daily transactions. Once you hit the limit, the pipeline throttles. The cap applies per pipeline, per day. Buying more capacity is possible but pricing scales steeply.
Infor Data Lake APIs have practical limits around request rates, payload sizes, concurrent queries, and Streaming Pipeline transaction caps. Many environments throttle a pipeline after roughly 165,000 transactions per day per pipeline. Large integrations usually require batching, pagination, and incremental sync strategies.
The most common workaround is distributing workloads across multiple pipelines and reducing unnecessary transactions through batching and incremental syncs. Many teams also move heavy bulk loads outside Streaming Pipelines into external ETL or database processes.
No. SqlBulkCopy is optimized for fast inserts only. Updates and deletes are typically handled afterward using staging tables with SQL MERGE, UPDATE, or DELETE statements.
Yes, if Mirrordex is built using supported interfaces such as Data Lake APIs, ION APIs, or Streaming Pipelines. Since multi-tenant SaaS environments do not allow direct database access, integrations usually rely on API-based synchronization methods.
Infor Streaming Pipelines cap at around 165,000 transactions per day. Here's why initial loads take so long and how to...
Engineering operational certainty
© 2026 Cesova. All rights reserved.