zodern

Meteor Up 1.5

Meteor Up is a cli tool to setup servers and deploy Meteor or other types of apps to them. After working on it off and on for over two years, version 1.5 is finally ready.

I have been using the betas for a few projects over the past 12 months, and by now it might be the most stable version of mup so far. Many of the bugs in Mup 1.4 have been fixed, and improvements have been made to automatically fix common mistakes in the mup config or show additional validation errors.

Performance #

Meteor Up 1.3 added a Prepare Bundle step to deploys which builds a docker image containing the app's bundle and rebuilt npm dependencies. This fixed many common issues new users had with Meteor Up, and made deploys more reliable and easier to troubleshoot. However, deploys were slower, especially for apps with large bundles.

Mup 1.5 adds the app.docker.useBuildKit option. When enabled, Meteor Up will use Build Kit, a new image builder in Docker. Using an experimental feature (RUN mounts), we are able to avoid the slower parts of building the image. For most apps I measured, this speeds up Prepare Bundle by 60% (one app took 1.4 minutes instead of 3.5). Since this does use experimental features, it is disabled by default.

Here is where you would add the useBuildKit option to your config:

module.exports = {
  app: {
    docker: {
      useBuildKit: true
    }
  }
}

The slowest parts of the deploy process done per server are uploading the bundle and running Prepare Bundle. Previously, when deploying to multiple servers, both steps had to be done for each server, one server at a time. Deployed times scaled almost linearly with the number of servers.

Mup 1.5 adds support for using a private docker registry. When configured, the app bundle will be uploaded to a single server where Prepare Bundle will be run. Afterwards, the image will be pushed to the private docker registry. All other servers will pull the image from the registry, skipping the upload and Prepare Bundle steps. Depending on the app, this will remove up to 90% of the increase to the deploy time when adding a server.

For example, to use the Google Cloud Container Registry create a json key, get your Google Cloud project id, and add this to your config:

module.exports = {
  // ...rest of config
  privateDockerRegistry: {
      host: 'https://gcr.io',
      imagePrefix: 'gcr.io/<project id>',
      username: '_json_key',
      password: '<json key file content>',
    },
  }
}

A few other improvements:

Load Balancing and Zero Downtime Deploys #

Mup can now set up load balancing using Nginx. Sticky sessions are enabled by default, but can be disabled for non-meteor apps or Meteor apps that do not use sockjs.

When load balancing is enabled and the app is deployed to at least two servers, there will be zero downtime when deploying. Mup deploys to one server at a time. While the app is being restarted on a server, Nginx detects that it is down and will send requests to a different server.

How it works:

  1. For the app to be available to nginx instances on other servers, it will be exposed to the internet (or to the private network if the new privateIp option is configured for the server). In the future, mup might setup a firewall to restrict access to the servers with nginx.
  2. To avoid conflicts with other apps, it uses a random port between 10,000 and 20,000. The port is always the same for the app as long as its name is not changed. mup validate --show can be used to view the port if you need to open it in the firewall.
  3. When mup setup or mup proxy setup is run, a Nginx config is generated with a list of the app's server's private IP addresses, or, if that isn't available, their host.

Production Debugging #

There is a new command, mup meteor debug <server name>, which enables the node inspector for your app. You can then connect to it locally using Chrome's DevTools for Node or any other Node debuggers. This can be used to take memory or CPU profiles, or to access the console. Breakpoints should not be used since they pause the server when hit, but logpoints are available. This feature has been very helpful as I've investigated performance issues the past couple of months.

Swarm Integration #

Mup is now able to create and manage a Docker Swarm cluster. In your mup config you describe what you want your cluster to look like, and Meteor Up will create or adjust the cluster to match. This feature is experimental as it could have large bugs or breaking changes.

The simplest config would be:

module.exports = {
  servers: {
    // list of servers
  },
  swarm: {
    enabled: true
  }
};

Then run mup setup. Mup will create the swarm cluster.

You might notice that there is no option to mark which servers are managers. Mup uses various criteria to pick the number of managers and which servers are managers for reliability, to meet the needs of plugins, and to minimize unnecessary changes. mup docker status can be used to see which servers are the managers. If you need more control over this, please create an issue.

The swarm cluster can be customized by:

Mup takes the information from the mup config and plugins, the existing cluster state, and some additional rules for avoiding problematic setups, and creates the desired configuration. It then checks the current status of the swarm cluster and makes any adjustments. mup docker status can be used to view some details on the swarm cluster.

There are a number of ways a swarm cluster could fail. Meteor Up tries to show helpful error messages where possible, or automatically fix issues. In the worse case scenario, the swarm cluster can be recreated by running mup docker destroy-cluster && mup setup && mup deploy.

Services #

When swarm is enabled, meteor apps are deployed as swarm services. To be accessible, the reverse proxy must be used. Load balancing and rolling deploys are always enabled.

If you want more flexibility, you do not have to use Meteor Up to deploy services. For one project I used mup to manage the cluster and node labels, and Docker Compose to deploy the services.

There are new apis for plugins to give Meteor Up the desired config for a service, and Meteor Up will compare it to the current service config to decide how to create or update it to match. The Meteor plugin uses this to deploy Meteor apps as services.

Future of Swarm Integration #

My plan was to eventually make using swarm the default once it had been stable for a while. Unfortunately, I've experience reliability issues and some bugs with Docker Swarm, and would not be comfortable for people unfamiliar with swarm to use it. Since I no longer use Swarm for my apps, I am looking for help to maintain the integration. If you are interested, please create an issue or let us know on Gitter.

Remaining issues:

New Website #

The Meteor Up website at https://meteor-up.com/ has been updated. The home page has been rewritten to better describe the features and benefits, and the docs were redesigned for better readability. Before and after screenshots are at https://github.com/zodern/meteor-up/pull/1151.

Updating #

The instructions to update are the same as any previous update. It should be completely backwards compatible.

# Update mup
npm i -g mup

# Update server setup
mup setup

# Update app config
mup reconfig

Mup 1.5 requires Docker 18 or newer. mup setup will update docker if needed.

What is Next #

Mup 1.6 will focus on polishing and cleaning up existing features and plugins. If time allows, I plan to also implement:

The goal is to release it in August or September. I will reduce the scope, if needed, to avoid taking another two years.

This year I plan to start work on version 2. Version 2 will likely:

Have a comment? Please Send me an email

Want More? Join my monthly newsletter for new blog posts and news about Meteor and my projects.

Have a Meteor app in production? I run Monti APM for production monitoring to help improve performance and reduce errors.