zodern

Implementing Flare - Single File Blaze Templates

While looking into integrating hot module replacement for Blaze, I decided to experiment with adding single file templates for Blaze, and seeing if we can make any other improvements to the developer experience. This post is the first of a series into how it can be implemented.

Initially, I was doing this for fun and to learn. I didn't expect the end result to be very useful as I haven't done much with Blaze for several years. However, as I work on this and talk to developers using Blaze, I am getting excited about the end result and think it will actually be useful. If not, my plan for this series is to give you the knowledge and examples needed to create your own solution that meets your needs.

Let's start. First, we will need to create a new package. Create a new folder named flare. Usually I have a folder on my computer named meteor-packages that I store local packages in. Then I set the environment variable METEOR_PACKAGE_DIRS=/path/to/meteor-packages so Meteor can find them.

Inside the flare folder, we want to create two files.

package.js:

Package.describe({
  name: 'flare',
  version: '1.0.0',
  summary: 'Single file components for Blaze',
  documentation: 'README.md',
});

Package.registerBuildPlugin({
  name: "compileBlaze",
  use: [
    'ecmascript@0.5.8',
    'templating-tools@1.1.2',
    'babel-compiler@7.5.3'
  ],
  npmDependencies: {
    'parse5': '6.0.1',
    'acorn': '8.0.4',
    'recast': '0.19.0',
    'ast-types': '0.14.2',
    'periscopic': '2.0.2',
    'globals': '13.3.0',
  },
  sources: [
    'compiler.js'
  ]
});

Package.onUse(function (api) {
  api.use('isobuild:compiler-plugin@1.0.0');
});

This is fairly standard for a Meteor build plugin. At the top we describe the package and provide various metadata. Next we register the build plugin. This lists any packages it uses, npm dependencies, and our files it runs. It is possible to have multiple build plugins in a package, each with its own name, dependencies, and sources.

To opt in to newer build features, in onUse you tell Meteor that the package uses specific isobuild packages. These packages don't actually exist, and are only used to enable new features. For example, here we are using the new compiler features added in Meteor 1.2.

You can learn more about package.js files in the Meteor docs.

We also need the compiler.js file mentioned in the package.js we just created. In it we want to register our compiler:


class Compiler {

}

Plugin.registerCompiler({
  extensions: ['blaze'],
  archMatching: 'web',
  isTemplate: true
}, () => new Compiler());

This tells Meteor we will compile any files with the .blaze extension, for example navigation.blaze. An app or package can only use one compiler for each extension. We also will only compile files used in the web arch's. If we wanted to support SSR we would change that.

isTemplate is an undocumented option. Meteor loads the compile result from templates before any other type of file. This allows you to reference the templates in your js (Template.<template name>) without worrying if it was loaded first. Apps that use a main module shouldn't see any difference with this option enabled since files are loaded in the order you import them.

When the compiler is created, we return a new instance of a Compiler class. This class will handle compiling the files. Many Meteor template compilers use the caching-html-compiler package instead of starting from scratch, but it doesn't provide enough flexibility for our needs.

Now we can create a new app and add the package:

meteor create flare-app
meteor add flare

Next, we need to decide what a .blaze file will look like. There are many possibilities. For this experiment I am going to go with this:

<script>
  // $state is a function to get or set the component's state
  // $parent is the same as using '..' in a template and
  // used to access the parent context
  import { $state, $parent } from 'flare';

  import { Articles } from './collections';
  import { goTo } from './router';

  // We can import other blaze files and use them as a custom html tag in our template
  import Nav from './nav.blaze';
  import User from './user.blaze';

  // The name of the template.
  // Is optional, and defaults to the file's path
  export const name = 'Counter';

  export function onRendered() {
    $state.set('count', 0);
  }

  // Any function defined or imported can be used as a helper or event handler
  function increase() {
    const current = $state.get('count');
    $state.set('count', current + 1);
  }

  function hasInteracted() {
    return $state.get('count') > 0;
  }
</script>

<!-- No template tag! -->

<Nav>
  <h1>Counter</h1>
  <User />
</Nav>


<h1>Hello</h1>
<p>{$state.get('count')}</p>
<button on:click={increase}>Increase</button>

{#unless hasInteracted()}
  <p>Please click the button </p>
{/unless}

{#if $state.get('count') % 2 === 0}
  <p>Count is even!</p>
{/if}

{#each item in $parent()}
  <div class="item {item.enabled ? 'enabled' : ''}">
    {item.name}
  </div>
{/each}

{#each article in Articles.find()}
  <p on:click="{() => goTo(`/article/${article._id}`)}">
    {article.name} - {article.author}
  </p>  
{/each}

<style>
  .item {
    padding-top: 10px;
  }

  .item.enabled {
    background: yellow;
  }
</style>

This looks quite a bit different from a normal blaze template.

  1. This is a single file template, so the js, css, and html for the template are in one file
  2. There is no template tag. Instead, everything is part of one template except the script and style tags
  3. We have a $state function to access a ReactiveDict unique to this instance of the template to store template state
  4. Instead of using Template., we export the functions or objects. This allows us to use the template api without needing to define the template name. It will also simplify implementing HMR.
  5. In the template, we only use a single bracket instead of two like spacebars/handlebars does ({count} instead of {{count}}). Svelte switched to using one to copy React, and hasn't encountered any major issues.
  6. The template tags (what spacebars calls {{ }}), can contain any js expression
  7. We add event handlers in the html
  8. Instead of registering helpers or event handlers, we can use any function imported or defined in the script
  9. Templates in .blaze files can be imported and used as an html element in the template instead of including by their name

By the end of these blog posts hopefully you will have the knowledge to implement your own ideas.

We will build up to compiling this. In this blog post, we will try to compile a very simplified version, and in later posts add more features until we can compile our full blaze file above.

<script>
  import { ReactiveVar } from 'meteor/reactive-var';

  export const name = 'counter';

  Template.counter.onCreated(function helloOnCreated() {
    this.counter = new ReactiveVar(0);
  });

  Template.counter.helpers({
    counter() {
      return Template.instance().counter.get();
    },
  });

  Template.counter.events({
    'click button'(event, instance) {
      // increment the counter when button is clicked
      instance.counter.set(instance.counter.get() + 1);
    },
  });
</script>

<button>Click Me</button>
<p>You've pressed the button {{counter}} times.</p>

This is much closer to a normal blaze component, and will be simpler to compile. You can add this to a new counter.blaze file in your app, and update your main.html and main.js files:

main.js:

import './main.html';
import './counter.blaze';

main.html:

<head>
  <title>flare-app</title>
</head>

<body>
  <h1>Welcome to Meteor!</h1>

  {{> counter}}
</body>

There are two options we have to compile this:

  1. Fork blaze and modify it to meet our needs
  2. Preprocess the blaze file and convert it into a normal template blaze can compile.

Since there is renewed interest lately in improving Blaze, we will go with option two so we can benefit from improvements to Blaze and easily remain compatible with the ecosystem.

How can we get our simplified blaze file to work with Blaze? First, we need to remove the script tag and its content so we can handle the js separately. Second, we have to wrap the html in a <template> tag and give it a name attribute. Then we simply have spacebars compile the template, and give Meteor the output and the js from the script tag.

As we implement more of the features, we will have to be able to understand the html and make changes to it. This requires parsing the content into its abstract syntax tree (AST). The AST is an object that represents the code in the file. We can then traverse this object to find the tags, attributes, and tag content in the file, and modify it. When working with AST, a useful tool is astexplorer.net. You can enter some code, pick a parser, and see what AST the parser produces for that code. We've already added parse5, a popular html parser, in our package.js file.

Now that we have a plan, we can start implementing our Compiler class. At the moment, it looks like this:

class Compiler {

}

To receive files to compile, a processFilesForTarget function should be added:

class Compiler {
  processFilesForTarget(files) {
    files.forEach((file) => {
      // Process and add the output.
      const output = compile(file.getContentsAsString());

      file.addJavaScript({
        data: output,
        path: file.getPathInPackage()
      });
    });
  }
}

processFilesForTarget is called once for the app, and once for app or package that depends on flare. It compiles the content, and adds the resulting js, html, and css.

We need to parse the content to extract the script tag and find the html that should be in the template. That can be accomplished with this:

const parse5 = require('parse5');

class Compiler {
  parseTags(file) {
    // Parse the file's content to get the AST
    const document = parse5.parseFragment(file.getContentsAsString());
    let scripts = [];

    // document.childNodes has all of the top-level tags in the file
    document.childNodes = document.childNodes.filter(tag => {
      if (tag.nodeName === 'script') {
        scripts.push(tag);
        return false;
      }

      return true;
    });

    if (scripts.length !== 1) {
      throw new Error(`File must have exactly one script tag in ${file.getPathInPackage()}`);
    }

    return {
      // The childNode is a text node with the script contents
      script: scripts[0].childNodes[0].value,
      template: parse5.serialize(document)
    };
  }

  // ... rest of class
}

Next, we need to parse the contents of the <script> tag to find the name export.

const acorn = require('acorn');
const { visit } = require('ast-types');

class Compile {
  parseJs(js, filePath) {
    const ast = acorn.parse(js, {
      sourceType: 'module',
      ecmaVersion: 'latest',
      allowImportExportEverywhere: true,
      locations: true,
      ranges: true
    });

    let templateName = null;

    visit(ast, {
      visitExportNamedDeclaration(path) {
        path.node.declaration.declarations.forEach(declaration => {
          const {
            id,
            init
          } = declaration;

          if (id.type !== 'Identifier' || id.name !== 'name') {
            return;
          }

          if (init.type !== 'Literal') {
            throw new Error(`name export must be a literal in ${filePath}`);
          }

          templateName = declaration.init.value;
        });

        return false;
      }
    });

    return {
      templateName
    };
  }

  // ... rest of class
}

We used acorn to create an AST of the js in the script tag. The options we pass it are fairly standard, except for allowImportExportEverywhere. This is not compliant with the ECMAScript specification, but Meteor allows it.

A common tool to navigate an AST is a visitor. Here we used the implementation provided by ast-types. We give it an options object with methods to visit specific types of nodes in the AST. To find which type of node you want, you can use astexplorer.net. Enter in some code in AST Explorer, and on the side you can find the node types. We want to find ExportNamedDeclaration nodes, which are created for export const name = 'template name'. Once we find them, we find their variable declaration for name and get its value.

Now we are ready to use these new methods to compile the file. Update processFilesForTarget to look like:

class Compiler {
  // ... rest of compiler

  processFilesForTarget(files) {
    files.forEach((file) => {
      // find the contents of the template and script
      const tags = this.parseTags(file);
      const {
        templateName
      } = this.parseJs(tags.script, file.getPathInPackage());

      const output = TemplatingTools.compileTagsWithSpacebars([{
        tagName: 'template',
        attribs: { name: templateName || `${file.getPackageName() || 'app'}/${file.getPathInPackage()}` },
        contents: tags.template,
        sourceName: file.getPathInPackage()
      }]);

      file.addJavaScript({
        data: `${output.js}\n${tags.script}`,
        path: file.getPathInPackage()
      });
    });
  }
}

It first parses the tags to get the template content and script content. We then parse the script content to find where we export the template name. We pass the template to TemplatingTools.compileTagsWithSpacebars to compile it, and add the js to our file.

Meteor doesn't allow adding multiple js files for one input file, so we merge the compiled template with the script tag's content. Normally this would be a bad idea since they could interfere with each other's scope, but the compiled template doesn't seem to use any local variables so it should be fine. If not, we can wrap it within its own closure.

You can view the compiled output by opening http://localhost:3000/app/app.js in a web browser, and searching for the file's name. If you try to load the app at this point in a web browser, it will crash with Uncaught SyntaxError: import declarations may only appear at top level of a module. We need to compile the script's content with Babel.

Let's add a constructor to create a new instance of the BabelCompiler and update processFilesForTarget to use it:

const os = require('os');

class Compiler {
  constructor() {
    this.babelCompiler = new BabelCompiler;
    this.babelCompiler.setDiskCacheDirectory(os.tmpdir() + '/meteor-babel-' + Math.random() + Date.now());
  }

  // ... rest of compiler

  processFilesForTarget(files) {
    files.forEach((file) => {
      // find the contents of the template and script
      const tags = this.parseTags(file);
      const {
        templateName
      } = this.parseJs(tags.script, file.getPathInPackage());

      const output = TemplatingTools.compileTagsWithSpacebars([{
        tagName: 'template',
        attribs: { name: templateName || `${file.getPackageName() || 'app'}/${file.getPathInPackage()}` },
        contents: tags.template,
        sourceName: file.getPathInPackage()
      }]);

      // Following code was modified to use babel
      const scriptContent = this.babelCompiler.processOneFileForTarget(
          file, tags.script
        ).data;

      file.addJavaScript({
        data: `${output.js}\n${scriptContent}`,
        path: file.getPathInPackage()
      });
    });
  }
}

You might notice we are setting the cache dir to a random temp folder. Babel's cache uses a hash of the source file's original content, not a hash of our modified content we are passing it. Because of this, if we change how we modify the content, Babel will continue using old cache entries since the original file content was not modified. Usually in a published package this is fine since Meteor resets the cache whenever we release a new version, but in development we are modifying the compiler without increasing the version after every change. Before publishing the package, we will want to change that line so cache will work. Later I will show how to properly implement caching.

Now if you visit the app, you will have a fully functioning counter.

The contents of the compiler.js file at this point look like:

const parse5 = require('parse5');
const acorn = require('acorn');
const { visit } = require('ast-types');
const os = require('os');

class Compiler {
  constructor() {
    this.babelCompiler = new BabelCompiler;
    this.babelCompiler.setDiskCacheDirectory(os.tmpdir() + '/meteor-babel-' + Math.random() + Date.now());
  }

  parseTags(file) {
    // Parse the file's content to get the AST
    const document = parse5.parseFragment(file.getContentsAsString());
    let scripts = [];

    // document.childNodes has all of the top-level tags in the file
    document.childNodes = bodyContent = document.childNodes.filter(tag => {
      if (tag.nodeName === 'script') {
        scripts.push(tag);
        return false;
      }

      return true;
    });

    if (scripts.length !== 1) {
      throw new Error(`File must have exactly one script tag in ${file.getPathInPackage()}`);
    }

    return {
      // The childNode is a text node with the contents
      script: scripts[0].childNodes[0].value,
      template: parse5.serialize(document)
    };
  }

  parseJs(js, filePath) {
    const ast = acorn.parse(js, {
      sourceType: 'module',
      ecmaVersion: 'latest',
      allowImportExportEverywhere: true,
      locations: true,
      ranges: true
    });

    let templateName = null;

    visit(ast, {
      visitExportNamedDeclaration(path) {
        path.node.declaration.declarations.forEach(declaration => {
          const {
            id,
            init
          } = declaration;

          if (id.type !== 'Identifier' || id.name !== 'name') {
            return;
          }

          if (init.type !== 'Literal') {
            throw new Error(`name export must be a literal in ${filePath}`);
          }

          templateName = declaration.init.value;
        });

        return false;
      }
    });

    return {
      templateName
    };
  }

  processFilesForTarget(files) {
    files.forEach((file) => {
      // find the contents of the template and script
      const tags = this.parseTags(file);
      const {
        templateName
      } = this.parseJs(tags.script, file.getPathInPackage());

      const output = TemplatingTools.compileTagsWithSpacebars([{
        tagName: 'template',
        attribs: { name: templateName || `${file.getPackageName() || 'app'}/${file.getPathInPackage()}` },
        contents: tags.template,
        sourceName: file.getPathInPackage()
      }]);

      const scriptContent = this.babelCompiler.processOneFileForTarget(
          file, tags.script
        ).data;

      file.addJavaScript({
        data: `${output.js}\n${scriptContent}`,
        path: file.getPathInPackage()
      });
    });
  }
}

Plugin.registerCompiler({
  extensions: ['blaze'],
  archMatching: 'web',
  isTemplate: true
}, () => new Compiler());

In the next post, we will look into parsing the template tags to support any js expression and be able to call functions in the script tag.

Have a comment? Please Send me an email

Want More? Join my monthly newsletter for new blog posts and news about Meteor and my projects.

Have a Meteor app in production? I run Monti APM for production monitoring to help improve performance and reduce errors.