LARA Deployment & App Migration for Dataddo

Client

Dataddo

As Dataddo was moving to a more mature product and application design, they were looking for a cloud native platform to run their applications, fast.

The Client

Dataddo is a cloud-based, no-code data integration and management platform that works with almost any online data service. The Dataddo app can transform and transfer data to a wide range of databases, DWH solutions, cloud storages and dashboards.

Challenge

Dataddo was looking for a cloud native platform they could trust and build their business around, including everything needed to run a cloud native and highly scalable application. Our LARA helped them get a platform which was:

Auto scalable
Autohealing
Highly available
Cost optimal
Modular
Observable

All of this was done with the emphasis on a quick time to market delivery.

‍

Technology Stack

AWS EKS, Aurora, ElastiCache, OpenSearch
Go, PHP, Vue.js
PosrgreSQL, MySQL, MongoDB, Redis, RabbitMQ
Terraform, Helm
GitlabCI, ArgoCD, Prometheus, Thanos, Grafana

Solution

To implement initial infrastructure architecture, deploy EKS with tested set of addons (external-dns, argocd, thanos, nginx-ingress controller, cert-manager, external-secrets, aws-loadbalancer-controller, metrics-server, log gathering solution) and migrate different refactored applications in to EKS with proper CI/CD, Monitoring, Automation & Backend Databases. To speed up this process we used our ready made and battle tested set of IaaC modules called Labyrinth Labs Reference Architecture.

%%15%%

microservices

%%100TB%%

data transferred monthly

Our Process

1.STEP

How we started [implementation]

Implementation of the following account structure with a separate audit account to make sure we can store and audit Cloudtrail logs outside of developer accessible accounts. We have also added 4 accounts for different types of workload:

Dev
- Dev workload
Test
- Test workload
Prod
- Production workload
Shared
- Used for services unrelated with customer workload e.g. monitoring, logging, CI/CD

Process [kubernetes]

Then, with our help Dataddo refactored their applications to work in the highly dynamic world of Kubernetes. Applications were modified to work with Kubernetes:

no stateful applications
using requests/limits to help application deployment scalability
highly available deployments with multiple replicas
observable infrastructure with multiple views
infrastructure metrics
application metrics
application logs

Process [cost optimization]

Implementation of different workload cost optimisations:

migrated to Spot instances
used cloud savings plan for workloads not running on spot instances
we haven’t used reserved instances because customer is using MongoDB

To better leverage Kubernetes Architecture we have helped the customer to move into a more automated Continuous Delivery model. Where they are using prepared helm charts to deploy application containers images into the cluster.

Continuous Integration pipeline is used for application verifications (test) and container image build.

Finish [utilization EKS]

Outcomes

By using our Reference Architecture and EKS Dataddo was able to scale their Workload fast and in a cost efficient way. Their existing architecture enables them to deliver more efficient software, faster and cheaper than their previous setup. It’s also much easier to maintain and support with a state of the art architecture and tools like Helm, Terraform, GitlabCI and Argo, which are already familiar among the developers.

Because Kubernetes is a highly dynamic and distributed system we made sure monitoring and observability stack were properly configured to gather as much information as possible about application/infrastructure behavior. This helps us and our customer deliver a much better SLA to their customers who are looking for a stable platform for their business needs.

‍

Labyrinth Labs Reference Architecture helped them design and implement better infrastructure patterns and migrated their applications to EKS.

During our work we have implemented:

CI/CD
Monitoring
Infrastructure as Code
Account structure
Application refactoring

Tomáš Sedláček

CTO

Dataddo

I’d definitely recommend them because they have strong knowledge in the infrastructure field, including AWS. Their team members are very responsive and able to help us when we need it.

Future steps [life-long support]

Labyrinth Labs provides long-term support for the newly built infrastructure, including regular upgrades and maintenance of the platform, as well as continuous improvements of the setup.