Platform Blueprints
Faster and More Productive Data Movement.
Add end-to-end dataflows & transformation rules – in minutes, not weeks – for faster data ROI.
Dataflow Integration Platform
Highly scalable data integration platform to make your analytics lifecycle faster and more productive.
Versatile Dataflow Management
Integrated data movement across sources with a drag-and-drop UI
Stream and/or Batch
Flexible blended data flows, both real-time and scheduled ingestion
Continuous Monitoring
Auto-recovery from source data problems for continuous data integrity
The Problem
What happens when data supply isn’t unleashing analytic demand?
Everyone wants a data-driven business; there seems to be more data every second. With so many ways to manipulate and store it, the path from data sources to useful business insight is getting slower, more fragmented and more siloed.
End-to-end Data sprawl
Reports and dashboards proliferate, compounding data lifecycle challenges and undermining re-use.
Frustrated Data Urgency
Inconsistent data completeness, freshness, reliability, and performance derails business confidence.
Flexibility leads to gridlock
Too many sources behave differently in quality, speed, and structure — accelerating technical debt.
The Solution
At CloudGeometry, we think there’s a better way
Designed to meet the dynamic demands of the modern data-driven business, the DataFlow Integration Platform by CloudGeometry provides a complete solution to data ingestion and delivery. Unlike conventional 20th century ETL tools, it’s designed from the bottom up to meet the dynamic needs of demanding analytics and data science workloads.
Built atop the StreamSets open source project (we are active contributors to the StreamSets upstream code) CloudGeometry closes the gap: flexible data intake, full lifecycle management, many-to-many data topologies, and production-grade data quality monitoring.
CloudGeometry solves for better data ingestion & delivery.
Key Features
DataFlow Integration Platform
Drag-and-drop pipeline canvas to wire SaaS APIs, and Social Media feeds, IoT and more to on-premises and cloud-based infrastructure
Continuous Incremental Change
Configure new data connectors with dozens of ready-to-go connectors preconfigured for popular APIs and data feeds
Data Ingestion
A powerful, modern data connector framework, designed to pull data from any source, stream or batch. Add new data & transformation rules in minutes, not weeks.
Data Transformation
Change data structures and types with functional expressions, within a data flow, and by batch-mode execution
Continuous data flow management
Get real-time data flow statistics for every stage and event trigger, plus exception-handling rules to re-process/recover from unexpected problems in source data
Cloud-native Control Hub
Management console to view and control pipeline running on premises, cloud or edge with real-time metrics for throughput, latency and error rates.
Still have questions? Get in touch
How we do it
Our DataFlow Integration Platform has been battle-tested by clients whose business relies on continuous ingest of 3rd party data. The reality is that these data inputs — feeds, APIs, 3d-party services — frequently change without notice, choking downstream dependencies.
We integrate production-grade monitoring with tight operational discipline to post fixes without delay. We’ve seen first-hand how quickly we can solve problems with own advanced connectors and management tools, so our clients always data keep flowing.
Daniil Yaroslavtsev
DevOps Lead, CloudGeometry
Unifying Data Flows
Drive data value faster with the full mix of sources and destinations that your analytics agenda demands. Transform and enrich records within the pipeline; create and fire rules triggered by events that meet fined-grained conditions. Easily process change capture data or transactional data for CRUD operations within pipeline segments.
DataFlow Management & Performance
Create a centralized point of control across all your data with a microservices architecture. A visual topology to maps across applications and environments. You get a single point of control for deploying, registering, starting, scaling, and stopping data flows, managing their performance and data integrity.
Mix/Match Data origins
Widest range of data stores and data engines, transactional, batch and real-time, structured, SaaS APIs, cloud and on-prem, spanning nearly endless data formats.
Data formats include Avro, Binary, Datagram, Delimited, Excel, JSON, Log, Protobuf, SDC Record, Text, Whole File, XML
Control Hub
Build and run execute large numbers of complex data flows at scale.
Local, global, and remote pipelines can be shared, exported and imported
DataFlow Triggers & Events
Kick off tasks in response to events that occur in a pipeline or propagate to additional pipelines.
Streamsets Expression Evaluator, Field Remover, TensorFlow Evaluator for ML, and more
Continuous Data Integrity
Detect drift in incoming data, to automatically create or alter corresponding data in transition.
Postgres SQL, Oracle, Hive metadata, JDBC, Redshift, Kinesis
Global governance for sensitive data
In-stream discovery of data in motion to implement data protection policies at the point of data ingestion.
Publish metadata to data governance tools such as Cloudera Navigator / Apache Atlas
Dataflow SLA Management
View real-time statistics about pipelines; examine samples of data being processed; create rules and alerts to track SLAs.
Consolidated or per individual stream (e.g. Kafka, Kinesis, MapR)
Flexible pipeline processing
Choose execution modes: standalone, cluster, or edge; create or test pipelines in development sandbox.
Kick off events driven by Amazon S3, Databricks, Email, JDBC Queries, Spark and more
Unit Testing
Integrated frameworks for automated unit testing for every programming language.
JUnit, NUnit, Cucumber, TestNG, Scalatest
Cluster Batch & Streaming
Cluster manager and a cluster application can spawn additional workers as needed.
Read data from a Kafka cluster, MapR cluster, HDFS, or Amazon S3