Over the past few months I’ve had the opportunity to film the following series of What is/How to videos with some excellent folks at VMware. The series concerns technologies aimed at more developer-friendly and code-centric application integration: cloud native data pipelines, distributed API gateways, and distributed API management.

What is a Cloud Native Data Pipeline? How to use Spring Cloud Data Flow to get started with Cloud Native Data Pipelines

Sabby Anandan (Product Manager, Spring Team, VMware) and I kick things off with a chat about the current hype around the term “cloud native” and some of the challenges of creating and managing increasingly data-intensive applications (ergo the need for data pipelines).

When I ask Sabby how he would define a “cloud native data pipeline,” he starts with a CNCF definition of cloud native as “loosely coupled systems that are resilient, manageable, and observable,” then applies this concept to data:

Now, imagine extending these types of patterns and principles to a series of data-intensive applications, where each application performs a data processing step. Given the building blocks in the data pipeline are designed for cloud deployment, we can build reliant and robust data-intensive architectures at scale.

We then discuss the advantages and challenges of orchestrating cloud native data pipelines and Sabby demonstrates how to use Spring Cloud Data Flow: a Spring project (also offered through VMware Tanzu) that does “microservice based streaming and batch data processing for Cloud Foundry and Kubernetes.”

Sabby also covers use cases for both Spring Cloud Task and Spring Cloud Stream applications, and somehow manages to incorporate no less than THREE applications (and their data) into his demo.

What is a Distributed API Gateway? How to get started with Spring Cloud Gateway for Kubernetes

Chris Sterling (Senior Product Line Manager, API Management, VMware) and I discuss some of the problems that arise as APIs become more numerous and organizations attempt to wrangle API sources built over the years (often with different languages, frameworks, and technologies). Add in app modernization initiatives, increased use of cloud infrastructure, and adoption of distributed architectures, and you have a potentially messy (and distributed) API sprawl.

The first half of the video focuses on how a distributed API gateway can help address API sprawl by providing a consistent and standardized API consumption experience (complete with developer self-service pathways); in the second half Chris spins up an instance of an Animal Rescue sample application (complete with cute puppies and kitties) and demonstrates Spring Cloud Gateway for Kubernetes. It is worth noting that I found this diagram (which appears towards the end of the video) particularly useful in illustrating some of the differences between distributed and centralized API gateways.

In fact, I like the above diagram so much that I am dropping a table version here for anyone accessing this post via screen reader:

Characteristics of a Developer-friendly Distributed API Gateway

	Distributed API Gateway	Centralized API Gateway
Developer Accessibility	API route configuration life cycle is managed by development teams	Centrally managed and no direct access for developers
Deployment Patterns	CI/CD friendly: Route configuration lives alongside application code	Admins/Ops update the route configurations on developer’s behalf
Security	Simpler SSO configuration for OIDC/LDAP/SAML backends	Complex centralized SSO configuration across disparate API needs
Advanced Network Topologies	Secure services behind Gateway with container-to-container routing	Forces connection from SaaS -> Services thus broadening attack surface

What is Distributed API Management? How to get started with API portal for VMware Tanzu

Chris returns for a second video that builds on our previous discussion of API gateways to address broader distributed API management issues.

One thing I like about this video is that Chris expands upon the challenges associated with two use cases mentioned in our earlier discussion: 1) internal API sharing across lines of business and 2) external API consumer access. We also talk API discovery and Open API standards, then wrap with a demo of API portal for VMware Tanzu.

Related Resources

• Spring Cloud Data Flow project page
• Spring Cloud Data Flow on GitHub
• Chris’s blog post announcing the general availability of Spring Cloud Gateway for Kubernetes
• Spring Cloud Gateway project page
• VMware Tanzu Spring Cloud Gateway landing page
• The Animal Rescue sample app featured in some of the demos
• Blog post: API Portal for VMware Tanzu is GA

Disclosure: The videos discussed in this post were sponsored by VMware (but this post was not).