Mesosphere Links To Key Supporting Systems For Container Use - InformationWeek
IoT
IoT
Cloud // Infrastructure as a Service
News
8/1/2016
10:05 AM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Mesosphere Links To Key Supporting Systems For Container Use

Mesosphere announced its Data Center Operating System now works with data and message management systems from Lightbend, Confluent, and DataStax.

9 Promising Cloud Security Startups To Watch
9 Promising Cloud Security Startups To Watch
(Click image for larger view and slideshow.)

Mesosphere's Data Center Operating System for container orchestration is being teamed up with key open source systems to build more complex microservice applications. The combination is designed to give developers the ability to use containers in state-aware systems, as well as stateless ones.

A stateful application can order updates to a database system, read messages in required sequence, or synchronize operations of interdependent applications. Stateless code can't, and that can make all the difference in the kinds of applications that can be built, said Florian Leibert, Mesophere cofounder and CEO.

Docker, CoreOS, and Mesosphere have acknowledged containers in their early days were essentially stateless packages of code, able to be constructed in a uniform manner and moved around by developers in predictable ways.

But Leibert claimed in an interview that Mesosphere is changing what containers can be used for as they become parts of more sophisticated applications. By implementing its DC/OS Enterprise Edition with one of three partners' platforms, containers can take on stateful operations in a combined system.

(Image: chuyu/iStockphoto)

(Image: chuyu/iStockphoto)

The partners are: The DataStax commercial version of the open source Cassandra NoSQL data management system; Confluent's version of open source Apache Kafka data streaming, the Confluent Platform; and the Lightbend integration of Apache Spark data-handling, Apache Akka messaging system, and Lightbend's own Lagom framework.

The combination is called the Lightbend Reactive Platform. Lightbend built the Lagom framework for Java-based microservices. It was previously known as Typesafe and renamed itself on Feb. 23.

Liebert said it's essential for a container orchestration system to be able to work with other systems for the containers to function in modern apps. Mesosphere's DC/OS can do so because it was built with "two-level scheduling," which allows DC/OS to maintain general container operations on a large computer cluster, but yields the specifics of workload scheduling to the relevant subsystem, such as Lightbend Reactive Platform, DataStax, or Confluent Platform, which have their own schedulers.

Mesosphere DC/OS is based on the open source Mesos container orchestrator. A UC Berkeley paper first published in 2010, "Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center," describes how its two-level scheduling was built in.

Leibert said such a capability is essential if containers are going to live up to their hype as a future critical component of applications. He called two-level scheduling a required element of the second phase of container operation, or Containers 2.0. "Container 1.0 systems, including Kubernetes and Docker Swarm, use a single monolithic scheduler," Leibert wrote in an Aug.1 blog post on the Mesosphere website.

The drawbacks of the so-called Container 1.0 world are that it's hard to deploy a stateful application for data synchronization and database use, among other things, he said.

Architects may do so by designing a separate cluster for a stateful service, then trying to get it to work with other elements of the application. Any upgrade to the stateful service will require the application to shut down and await a restart, he added.

"If you wanted to add a Kafka node to a cluster to get it to scale up, you have to stop the application, provision the node, and restart the application," he said.

Kafka, on the hand, was architected to communicate with the open source Mesos container orchestrator, which sits at the heart of Mesosphere's DC/OS. DC/OS can start a cluster of a 1,000 servers, and then follow a systems administrator's directions on which subsystems to prioritize on it.

Those subsystems might include Cassandra, Kafka, and Spark. Provided they're supplied by DataStax, Confluent Platform, and Lightbend Reactive Platform, the DC/OS cluster operating system would be able to hand off scheduling of workloads to each, depending on its priority.

[Want to learn who's backing Mesosphere? Read Microsoft, HP Help Pump $73 Million Into Mesosphere Container Management.]

"If Spark was being used for Server log analysis, that might be a lower priority than website clickstream analysis," and could be scheduled accordingly through DC/OS supervision and handoffs.

Since Spark is a batch process, the log file analysis could be run during a slack period, with the virtual servers returned to the cluster as soon as traffic picked up again, said Leibert in the interview.

Administrators may set up a similar system with products other than DC/OS, but Leibert claimed they'll have to do work and have fewer automated functions to rely upon.

Mesosphere is trying to make it easy to add nodes to a cluster or containerized microservices to an application, he said. It also wants a system on a cluster to survive a node failure without losing track of the application's state.

The two-level scheduling "is a big differentiator for Mesos," he said, and by extension, Mesosphere. It can scale over a cluster of tens of thousands of servers. "No other system has the scalability of Mesos" at this time, he claimed.

Stateful container deployment is currently in use at Verizon, Esri, Autodesk, Wdellframe, and Time Warner Cable.

Charles Babcock is an editor-at-large for InformationWeek and author of Management Strategies for the Cloud Revolution, a McGraw-Hill book. He is the former editor-in-chief of Digital News, former software editor of Computerworld and former technology editor of Interactive ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
8/1/2016 | 7:30:24 PM
Mesosphere has roots in Twitter, Airbnb
CEO Liebert was the technology lead in bulding Airbnb's infrastructure, using Mesos. Benjamin Hindman, co-founder of the original Mesos project at Berkeley and co-founder of Mesosphere, was one of the leaders in creating the Mesos-based infrastructure at Twitter.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
8/1/2016 | 6:44:11 PM
An example of two-level scheduling
A key part of the white paper referred to: Mesos developers could have centralized task scheduling in Mesos:  "Instead, Mesos takes a different approach: delegating control over scheduling to the frameworks. This is accomplished through a new abstraction, called a resource offer, which encapsulates a bundle of resources that a framework can allocate on a cluster node to run tasks. Mesos decides how many resources to offer each frame-work, based on an organizational policy such as fair sharing, while frameworks decide which resources to accept and which tasks to run on them." Benjamin Hindman, Mesosphere co-founder, is one of the authors.

 

 

 
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
Digital Transformation Myths & Truths
Transformation is on every IT organization's to-do list, but effectively transforming IT means a major shift in technology as well as business models and culture. In this IT Trend Report, we examine some of the misconceptions of digital transformation and look at steps you can take to succeed technically and culturally.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll