Package detail

broccoli

broccolijs713.5kMIT3.5.2

Fast client-side asset builder

asset, browser, build, builder

readme

Broccoli

Build Status

A fast, reliable asset pipeline, supporting constant-time rebuilds and compact build definitions. Comparable to the Rails asset pipeline in scope, though it runs on Node and is backend-agnostic.

For more information and guides/documentation, checkout broccoli.build

For background and architecture, see the introductory blog post.

For the command line interface, see broccoli-cli.

Installation

npm install --save-dev broccoli
npm install --global broccoli-cli

Brocfile.js

A Brocfile.js file in the project root contains the build specification. It should export a function that returns a tree. Note: the Brocfile historically could export a tree/string directly, however this is now deprecated in favor of a function that can receive options

A tree can be any string representing a directory path, like 'app' or 'src'. Or a tree can be an object conforming to the Plugin API Specification. A Brocfile.js will usually directly work with only directory paths, and then use the plugins in the Plugins section to generate transformed trees.

The following simple Brocfile.js would export the app/ subdirectory as a tree:

export default () => 'app';

With that Brocfile, the build result would equal the contents of the app tree in your project folder. For example, say your project contains these files:

app
├─ main.js
└─ helper.js
Brocfile.js
package.json
…

Running broccoli build the-output (a command provided by broccoli-cli) would generate the following folder within your project folder:

the-output
├─ main.js
└─ helper.js

Options

The function that is exported from module.exports is passed an options hash by Broccoli that can be used when assembling the build.

The options hash is populated by the CLI environment when running broccoli build or broccoli serve. It currently only accepts a single option --environment, which is passed as env in the options hash.

Additionally --prod and --dev are available aliases to --environment=production and --environment=development respectively.

  • options:
    • env: Defaults to development, and can be overridden with the CLI argument --environment=X

For example:

export default (options) => {
    // tree = ... assemble tree

    // In production environment, minify the files
    if (options.env === 'production') {
        tree = minify(tree);
    }

    return tree;
}

TypeScript Support

A Brocfile.ts can be used in place of a Brocfile.js and Broccoli will automatically parse this through ts-node to provide TypeScript support. This allows developers to leverage type information when assembling a build pipeline. By default, Broccoli provides type information for the options object passed to the build function.

import { BrocfileOptions } from 'broccoli';

export default (options: BrocfileOptions) => {
  // tree = ... assemble tree

  // In production environment, minify the files
  if (options.env === 'production') {
    tree = minify(tree);
  }

  return tree;
};

Typescript by default only allows the ES6 modules import/export syntax to work when importing ES6 modules. In order to import a CommonJS module (one that uses require() or module.exports, you must use the following syntax:

import foo = require('foo');

export = 'bar';

You'll note the syntax is slightly different from the ESM syntax, but reads fairly well.

Using plugins in a Brocfile.js

The following Brocfile.js exports the app/ subdirectory as appkit/:

// Brocfile.js
import Funnel from 'broccoli-funnel';

export default () => new Funnel('app', {
  destDir: 'appkit'
})

Broccoli supports ES6 modules via esm for Brocfile.js. Note, TypeScript requires the use of a different syntax, see the TypeScript section above.

You can also use regular CommonJS require and module.exports if you prefer, however ESM is the future of Node, and the recommended syntax to use.

That example uses the plugin broccoli-funnel. In order for the import call to work, you must first put the plugin in your devDependencies and install it, with

npm install --save-dev broccoli-funnel

With the above Brocfile.js and the file tree from the previous example, running broccoli build the-output would generate the following folder:

the-output
└─ appkit
   ├─ main.js
   └─ helper.js

Plugins

You can find plugins under the broccoli-plugin keyword on npm.

Using Broccoli Programmatically

In addition to using Broccoli via the combination of broccoli-cli and a Brocfile.js, you can also use Broccoli programmatically to construct your own build output via the Builder class. The Builder is one of the core APIs in Broccoli, and is responsible for taking a graph of Broccoli nodes and producing an actual build artifact (i.e. the output usually found in your dist directory after you run broccoli build). The output of a Builder's build method is a Promise that resolves when all the operations in the graph are complete. You can use this promise to chain together additional operations (such as error handling or cleanup) that will execute once the build step is complete.

By way of example, let's assume we have a graph of Broccoli nodes constructed via a combination of Funnel and MergeTrees:

// non Brocfile.js, regular commonjs
const Funnel = require('broccoli-funnel');
const MergeTrees = require('broccoli-merge-trees');

const html = new Funnel(appRoot, {
  files: ['index.html'],
  annotation: 'Index file'
})

const js = new Funnel(appRoot, {
  files: ['app.js'],
  destDir: '/assets',
  annotation: 'JS Files'
});

const css = new Funnel(appRoot, {
  srcDir: 'styles',
  files: ['app.css'],
  destDir: '/assets',
  annotation: 'CSS Files'
});

const public = new Funnel(appRoot, {
  annotation: 'Public Files'
});

const tree = new MergeTrees([html, js, css, public]);

At this point, tree is a graph of nodes, each of which can represent either an input or a transformation that we want to perform. In other words, tree is an abstract set of operations, not a concrete set of output files.

In order to perform all the operations described in tree, we need to do the following:

  • construct a Builder instance, passing in the graph we constructed before
  • call the build method, which will traverse the graph, performing each operation and eventually writing the output to a temporary folder indicated by builder.outputPath

Since we typically want do more than write to a temporary folder, we'll also use a library called TreeSync to sync the contents of the temp file with our desired output directory. Finally, we'll clean up the temporary folder once all our operations are complete:

const { Builder } = require('broccoli');
const TreeSync = require('tree-sync');
const MergeTrees = require('broccoli-merge-trees');
// ...snip...
const tree = new MergeTrees([html, js, css, public]);

const builder = new Builder(tree);

const outputDir = 'dist';
const outputTree = new TreeSync(builder.outputPath, outputDir);

builder.build()
  .then(() => {
    // Calling `sync` will synchronize the contents of the builder's `outPath` with our output directory.
    return outputTree.sync();
  })
  .then(() => {
    // Now that we're done with the build, clean up any temporary files were created
    return builder.cleanup();
  })
  .catch(err => {
    // In case something in this process fails, we still want to ensure that we clean up the temp files
    console.log(err);
    return builder.cleanup();
  });

Running Broccoli, Directly or Through Other Tools

Helpers

Shared code for writing plugins.

Plugin API Specification

See docs/node-api.md.

Also see docs/broccoli-1-0-plugin-api.md on how to upgrade from Broccoli 0.x to the Broccoli 1.x API.

Security

  • Do not run broccoli serve on a production server. While this is theoretically safe, it exposes a needlessly large amount of attack surface just for serving static assets. Instead, use broccoli build to precompile your assets, and serve the static files from a web server of your choice.

Get Help

  • IRC: #broccolijs on Freenode. Ask your question and stick around for a few hours. Someone will see your message eventually.
  • Twitter: mention @jo_liss with your question
  • GitHub: Open an issue on a specific plugin repository, or on this repository for general questions.

License

Broccoli was originally written by Jo Liss and is licensed under the MIT license.

The Broccoli logo was created by Samantha Penner (Miric) and is licensed under CC0 1.0.

changelog

master

v3.5.2 (2021-05-03)

:bug: Bug Fix

Committers: 1

v3.5.1 (2021-02-17)

:bug: Bug Fix

v3.5.0 (2020-12-07)

:rocket: Enhancement

  • #474 add the ability to ignore absolute paths from watcher (@ef4)

:bug: Bug Fix

  • #473 remove vestigial filter option (@ef4)

:memo: Documentation

  • #470 Fix wrong package name in README (@ursm)

Committers: 2

  • Edward Faulkner (@ef4)
  • Keita Urashima (@ursm)

3.4.2

  • fix TypeScript types
  • add missing semi colon

3.4.1

  • Revert retry feature (caused downstream failures for ember-cli users) (#459)

3.4.0

  • [Feature] stop current build, when rebuild is triggered (#408)

3.3.3

  • [BUGFIX] Ensure buildAnnotation survives a round trip from watcher -> builder -> the fulfillment value of watcher.currentBuild

3.3.2

  • Semver compatible dependency upgrades

3.3.1

  • [BUGFIX] restore static APIs on Broccoli.Builder that got dropped during the typescript conversion

3.3.0

  • Code-base utilizes async/await where appropriate
  • [BUGFIX] no longer cause spurious unhandled promise rejections warnings
  • convert lib to typescript
  • migrate to github actions

3.2.0

  • Add input node change tracking (#419)
  • Initial typescript conversion (#422)
  • Fixup cli usage information (#421)
  • Use a more appropriate data structure for nodeWrappers (#418)
  • Support serving over HTTPS (#417)

3.1.2

  • Enable colored syntax highlighting in error messages from babel (#415)

  • [BUGFIX] Add annotation object for broccoli-sane-watcher compatibility

3.1.0

  • [Feature] use console-ui

3.0.1

  • [BUGFIX] ensure parity broccoli-sane-watcher the filePath property in the change event and the additional console logging from a sane event.

3.0.0

  • Add rebuild memoization support behind feature flag (#396)
    • BROCCOLI_ENABLED_MEMOIZE=true to turn on
  • Add more watcher events (#398)
  • Drop support for unsupported Node versions (#400)
    • Drop Node 6 support.
  • Bump broccoli-node-info to v2.0.0
  • Pass watchedNodes to Watcher/WatcherAdapter (#403)
    • Require watchedNodes arguments to Watcher/WatcherAdapter (#405)

2.3.0

  • Add Brocfile.ts TypeScript support (#390)

2.2.0

  • Cleanup syntax (#383)
  • Ensure builder.cleanup() waits on pending work (#393)

2.1.0

  • Add support for ES Modules syntax (#385)
  • Add support for Export function (#386)
  • Add support for Environment flag (#387)

2.0.1

  • Fix various issues resulting in out of memory errors during instrumentation node traversal.

2.0.0

  • Update sane to ensure no native dependencies are needed.
  • Cleanup Builder.prototype.build to properly return Promise<void> (removing the outputNodeWrapper).
  • Add documentation for programmatic Builder usage.
  • Update internal dependencies to latest versions.
  • Remove usages of RSVP (in favor of using native promises).
  • Add support for Node 10.
  • Drop support for Node versions older than 6.
    • Drop Node 4 support.
    • Drop Node 0.12 support.
  • Add visualization support via heimdalljs.
  • Ensure that mid-build cancelation avoids extra work.
  • Add --overwrite option to the command line interface which clobbers any existing directory contents with the contents of the new build.
  • Add --cwd option to the command line interface which allows customizing the builders working directory (and where the Brocfile.js is looked up from).
  • Add --output-path option to the command line interface.
  • Add --watch option to build sub-command.
  • Add --no-watch option to serve sub-command.
  • Add --watcher option to allow configuration of the watcher to be used. Currently supported values are polling, watchman, node, events.
  • General code cleanup and modernization.

1.1.4

  • Roll back broccoli-slow-trees dependency

1.1.3

  • Update dependencies

1.1.2

  • Update findup-sync dependency

1.1.1

  • Fix option parsing for --port

1.1.0

  • Add needsCache to pluginInterface. Allows opting out of cache directory creation.

1.0.0

  • Release without change

1.0.0-beta.8

  • Builder throws an error when a watched input directory is missing
  • Rework watcher
  • Pull broccoli-sane-watcher functionality into core
  • Update findup-sync dependency

1.0.0-beta.7

  • Remove wrong postinstall hook. This removes a spurious dependency on multidep.

1.0.0-beta.6

  • Add build event to watcher

1.0.0-beta.5

  • Remove redundant beginBuild/endBuild (formerly start/end) events on builder

1.0.0-beta.4

  • Improve test suite

1.0.0-beta.3

  • Minor cosmetic changes

1.0.0-beta.2

  • Add watcher.watch() method. Watcher no longer automatically starts watching; instead, you must call this method explicitly. It returns a promise that is fulfilled if you later call watcher.quit(), or rejected if watching one of the source directories fails.

    • server will call watcher.watch() for you.
    • In contrast, getMiddleware expects a watcher that is already watching.

1.0.0-beta.1

  • Drop support for plugins that implement only the old .read/.rebuild API
  • Fail build when a source node is a file rather than a directory
  • Fail build when a source node doesn't exist
  • Builder API changes:

    • new Builder has a tmpdir option, which defaults to os.tmpdir() (typically /tmp); pass { tmpdir: './tmp' } to get the old behavior
    • .build() no longer returns a promise to the output path; instead, the output path stored at builder.outputPath and doesn't change between builds
    • start, end, nodeStart, nodeEnd events renamed to beginBuild, endBuild, beginNode, endNode
    • Nodes passed to nodeBegin/nodeEnd arguments are "node wrapper" objects (also accessible at builder.nodeWrappers); timings now reside at nodeWrapper.buildState.selfTime/totalTime and are in milliseconds, not nanoseconds
    • build() no longer takes a willReadStringTree callback argument; instead, source directories are recorded at builder.watchedPaths
  • Watcher API changes:

    • Add watcher.quit() method, which returns a promise until a running build has finished (if any)
    • Rename watcher.current to watcher.currentBuild, and remove watcher.then
    • Use RSVP.EventTarget instead of EventEmitter for events
  • Build error objects have been changed to Builder.BuildError objects, which contain additional information at err.broccoliPayload

0.16.8

  • Add builder hooks

0.16.7

  • Export watcher and middleware as Watcher and getMiddleware

0.16.6

  • Export watcher and middleware

0.16.5

  • On BROCCOLI_WARN_READ_API=y, print deprecation warning for .rebuild as well

0.16.4

  • Return server objects for easier extensibility

0.16.3

  • Do not silently swallow errors in change/error event handlers

0.16.2

  • Add missing dependency

0.16.1

  • Add Node interface to Builder, to enable building visualizations
  • Export Builder.getDescription(tree) helper function
  • Add footer to directory listings, so people know where they come from

0.16.0

  • Remove built-in LiveReload server; tools like Ember CLI inject LiveReload scripts, which is generally preferable because it doesn't need a separate port

0.15.4

  • Send Cache-Control header for directory listings and redirects
  • Honor liveReloadPath middleware option in directory listings as well
  • Add autoIndex middleware option to disable directory listings

0.15.3

  • Correctly display multi-line error messages

0.15.2

  • Add ability to inject live-reload script into error messages

0.15.1

  • Hide API warnings behind $BROCCOLI_WARN_READ_API env flag
  • Add support for new error API
  • Fail fast if build output directory already exists

0.15.0

  • Print deprecation warnings for plugins only providing old .read API

0.14.0

  • Add support for new .rebuild API, in addition to existing (now deprecated) .read API

0.13.6

0.13.5

  • Add missing var

0.13.4

  • More detailed error message when a tree object is invalid
  • Watcher no longer rebuilds forever when a very early build error occurs

0.13.3

  • Fix SIGINT/SIGTERM (Ctrl+C) handling to avoid leaking tmp files

0.13.2

  • Extract slow trees printout into broccoli-slow-trees package
  • Allow the tree cleanup method to be asynchronous (by returning a promise).

0.13.1

0.13.0

  • Dereference symlinks in broccoli build output by copying the files or directories they point to into place
  • Sort entries when browsing directories in middleware

0.12.3

  • Exclude logo and test directories from npm distribution

0.12.2

  • Fix directory handling in server on Windows

0.12.1

  • Show directory listing with broccoli serve when there is no index.html

0.12.0

  • Add willReadStringTree callback argument to Builder::build and retire Builder::treesRead
  • Update Watcher and Builder interaction to prevent double builds.
  • Avoid unhandled rejected promise
  • Fix trailing slash handling in server on Windows

0.11.0

  • Change Watcher's change event to provide the full build results (instead of just the directory).
  • Add slow tree logging to broccoli serve output.
  • Add logo

0.10.0

  • Move process.exit listener out of builder into server
  • Change Builder::build() method to return a { directory, graph } hash instead of only the directory, where graph contains the output directories and timings for each tree
  • Avoid keeping file streams open in server, to fix EBUSY issues on Windows

0.9.0

  • Brocfile.js now exports a tree, not a function (sample diff)

0.8.0

  • Extract bowerTrees into broccoli-bower plugin (sample diff)

0.7.2

  • Update dependencies

0.7.1

  • Do not use hardlinks in bower implementation

0.7.0

  • Remove broccoli.MergedTree; it has been extracted into broccoli-merge-trees (sample diff)

0.6.0

  • Disallow returning arrays from Brocfile.js, in favor of broccoli-merge-trees plugin (sample diff)

0.5.0

  • Remove broccoli.makeTree('foo') in favor of string literals (just 'foo') (sample diff)
  • Remove broccoli.Reader
  • Add --version command line option

0.4.3

  • Correct mis-publish on npm

0.4.2

  • Preserve value/error on Watcher::current promise
  • This version has been unpublished due to a mis-publish

0.4.1

  • Extract broccoli.helpers into broccoli-kitchen-sink-helpers package

0.3.1

  • Report unhandled errors in the watcher
  • Add support for .treeDir property on error objects
  • Improve watcher logic to stop double builds when build errors happen

0.3.0

  • Bind to localhost instead of 0.0.0.0 (whole wide world) by default

0.2.6

  • Overwrite mis-pushed release

0.2.5

  • Refactor watcher logic to use promises
  • Turn the hapi server into a connect middleware

0.2.4

  • Use smaller bower-config package instead of bower to parse bower.json files

0.2.3

  • Add --port, --host, and --live-reload-port options to serve command

0.2.2

  • Update hapi dependency to avoid file handle leaks, causing EMFILE errors

0.2.1

  • In addition to Brocfile.js, accept lowercase brocfile.js
  • Fix error reporting for string exceptions

0.2.0

  • Rename Broccolifile.js to Brocfile.js
  • Change default port from 8000 to 4200

0.1.1

  • Make tree.cleanup non-optional
  • Rename broccoli.read to broccoli.makeTree

0.1.0

  • Bump to indicate beta status
  • Remove unused helpers.walkSync (now in node-walk-sync)

0.0.13

  • Extract Transformer into broccoli-transform package (now "Transform")
  • Extract Filter into broccoli-filter package

0.0.12

  • In plugin (tree) API, replace .afterBuild with .cleanup
  • Move temporary directories out of the way

0.0.11

  • Extract factory.env into broccoli-env package
  • Eliminate factory argument to Broccolifile

0.0.10

  • Change to a .read-based everything-is-a-tree architecture
  • Various performance improvements
  • Various plugin API changes
  • Add MergedTree
  • Broccolifile may now return an array of trees, which will be merged
  • Expose broccoli.bowerTrees(), which will hopefully be redesigned and go away again
  • Remove Component base class
  • Remove CompilerCollection and Compiler base class; use a Transformer
  • Remove Tree::addTransform, Tree::addTrees, and Tree::addBower
  • Builder::build now has a promise interface as well

0.0.9

  • Expect a Tree, not a Builder, returned from Broccolifile.js

0.0.8

  • Fold Reader into Tree
  • Replace PreprocessorPipeline and Preprocessor with Filter; each Filter is added directly on the tree or builder with addTransform

0.0.7

  • Bind to 0.0.0.0 instead of localhost
  • Add factory.env based on $BROCCOLI_ENV
  • Do not fail on invalid Cookie header
  • Use promises instead of callbacks in all external APIs

0.0.6

  • Here be dragons