Using Phoenix Without Webpack

Functional programming with Elixir and building web apps with the Phoenix framework is some of the most fun I’ve had coding in years. Not only is Elixir a clean and well-designed language, Phoenix takes an HTML-over-the-wire approach similar to traditional frameworks like Django and Ruby on Rails–which I prefer over the client-side SPA/server-side API approach.

However, the one complaint I have with Phoenix is it pushes you towards using Webpack as a build tool for static assets. Since we’re no longer building heavy single-page applications, I would prefer to keep things simple by using native implementions of SASS for CSS preprocessing and ESBuild for JS bundling. With these tools working alongside each other, we can have a speedy little asset pipeline free of clunky Javascript tooling.

So how can we integrate these tools into an asset pipeline that works smoothly with Phoenix? Running the standalone CLI apps in separate terminal tabs is always an option, but that can get tedious with live reloading, digests, releases, and other mix tasks. I’ve finally come across a solution that works well with a bit of extra configuration.

System Dependencies

  • Elixir v1.11 and Phoenix v1.5
  • SASS installed globally (without npm to get the faster Dart implementation)
  • ESBuild installed globally

This tutorial assumes a basic knowledge of Elixir/Phoenix projects. If you’re brand new, pause here and jump over to their excellent getting started guide.

Application Setup

Create a new Phoenix application called foobar, specifying --no-webpack to prevent the Webpack boilerplate from being included. I’m also specifying --no-ecto so we don’t have to deal with a database in this example.

$ mix phx.new foobar --no-webpack --no-ecto

* creating foobar/config/config.exs
* creating ...

Fetch and install dependencies? [Yn] Y

Start your new application by going into the foobar directory and running mix phx.server. localhost:4000 should now be live with a lovely getting started page.

Let’s take a look at the boilerplate code that was generated—specifically the priv/static folder. This is where all of our static assets live, including all the CSS and JS.

priv/
	static/
		css/
		images/
		js/
		favicon.ico
		robots.txt

If we opted into using Webpack, this static folder would contain the generated output files while the source files would be found in assets/. Since we opted-out of the recommended build tool, these are the source files. Which is why we need to comment out priv/static in the gitignore file.

.gitignore
# Since we are building assets from assets/,
# we ignore priv/static. You may want to comment
# this depending on your deployment strategy.
# /priv/static/

Asset Pipeline

With our shiny new application setup and ready to go, we need to set up an asset pipeline from scratch. There’s four things our pipeline needs to do:

  • Transpile CSS files
  • Transpile Javascript files
  • Run commands in watch mode for the dev server
  • Run commands in build/minification mode for releases

Let’s start off by moving the boilerplate CSS and JS into assets/, where they can be treated as pre-transpiled source files (or you can create new files from scratch and delete these ones). Make sure you rename the .css extensions to .scss.

  • priv/static/css/app.css –> assets/css/app.scss
  • priv/static/css/phoenix.css –> assets/css/phoenix.scss
  • priv/static/js/app.js –> assets/js/app.js
  • priv/static/js/phoenix.js –> assets/js/phoenix.js

Our new assets/ directory should now look like this:

assets/
	css/
		app.scss
		phoenix.scss
	js/
		app.js
		phoenix.js

Let’s also update the import in app.scss from @import "./phoenix.css"; to @import "./phoenix";. SASS will automatically infer the file extension.

Next up is the watch scripts. We want Phoenix to run build commands for both of these asset types in watch mode when it starts the dev server, and live reload any changes. This is where the watchers runtime config option can be used in config/dev.exs. This allows us to specify a keyword list of shell commands to run alongside the server (this is why we need SASS and ESBuild installed as globals). We already have an empty list waiting for us on line 14, so let’s add our two watchers and output the results back into priv/static/css and priv/static/js.

config/dev.exs
config :foobar, FoobarWeb.Endpoint,
	http: [port: 4000],
	debug_errors: true,
	code_reloader: true,
	check_origin: false,
	watchers: [
		sass: [
			"--watch",
			"assets/css/app.scss",
			"priv/static/css/app.css"
		],
		esbuild: [
			"--watch",
			"--bundle",
			"--sourcemap",
			"--target=es2016",
			"--outfile=priv/static/js/app.js",
			"assets/js/app.js"
		]
	]

Now restart the server by hitting ctrl+c twice, followed by mix phx.server again. You’ll find the transpiled assets alongside their source maps back in place within priv/static. If you visit localhost:4000, everything still looks the same. Try adding some CSS and watch it live reload almost instantly.

assets/css/app.scss
@import "./phoenix";

$theme: lightblue;

body {
	background: $theme;
}

header {
	background: $theme;
	border: none;
}

.phx-hero {
	background: darken($theme, 10%);
	border-color: darken($theme, 15%);
}

We can even use modern Javascript features such as import, and ESBuild will speedily take care of the transform so it’s browser-compatible.

assets/js/app.js
import "./phoenix"
import "../../deps/phoenix_html/priv/static/phoenix_html"

const name = "jsonmaur"
console.log(`greetings, ${name}`)

Since priv/static/css and priv/static/js are now generated rather than being source files, let’s add both of them to the gitignore.

.gitignore
# Since we are building assets from assets/,
# we ignore priv/static. You may want to comment
# this depending on your deployment strategy.
# /priv/static/
/priv/static/css/
/priv/static/js/

Zombie Script

We have a slight problem. If you stop the Phoenix server again, the sass and esbuild processes don’t properly terminate along with it. Peek at your Activity Monitor and you’ll see the old processes start to build up as you start/stop the server over and over again. Until you manually stop these processes or restart your computer, they’ll continue to build up.

This is because those processes are still watching for file changes, and it’s a known issue with the Elixir Port module referred to as zombie operating system processes. The docs gives us a very handy script that can ensure these zombie processes are terminated along with the VM, so let’s save that to zombie.sh in the project root and run chmod +x zombie.sh to ensure it’s executable.

zombie.sh
#!/usr/bin/env bash

# Start the program in the background
exec "$@" &
pid1=$!

# Silence warnings from here on
exec >/dev/null 2>&1

# Read from stdin in the background and
# kill running program when stdin closes
exec 0<&0 $(
	while read; do :; done
	kill -KILL $pid1
) &
pid2=$!

# Clean up
wait $pid1
ret=$?
kill -KILL $pid2
exit $ret

Now jump back into config/dev.exs and prefix the zombie script to both commands. Since the script path needs to be expanded and used more than once, we’ll save it to a variable. watchers is a keyword list so we can use tuples with the first element containing the expanded path, and the second element containing a list of arguments.

config/dev.exs
zombie = Path.expand("../zombie.sh", __DIR__)

config :foobar, FoobarWeb.Endpoint,
	http: [port: 4000],
	debug_errors: true,
	code_reloader: true,
	check_origin: false,
	watchers: [
		{zombie,
		 [
			 "sass",
			 "--watch",
			 "assets/css/app.scss",
			 "priv/static/css/app.css"
		 ]},
		{zombie,
		 [
			 "esbuild",
			 "--watch",
			 "--bundle",
			 "--sourcemap",
			 "--target=es2016",
			 "--outfile=priv/static/js/app.js",
			 "assets/js/app.js"
		 ]}
	]

No more zombie processes when we run the server. If you manually stop those rogues who are hanging around from before, you won’t see any new ones accumulating. All the watcher processes will terminate properly.

Custom Mix Task

This is all we need for local development, so start building to your heart’s content. But when the time of reckoning comes and you need to deploy a new release, you’ll find the watcher scripts don’t work so well. We need a way to run SASS and ESBuild outside of the server context, without watch mode, with minification enabled.

The steps for releasing and deploying a Phoenix app can be found here. We essentially need to compile the source code, digest the static assets, create a cache manifest, and run the server in production mode. In the spirit of avoiding NPM, we’ll create a custom mix task that can easily be integrated with the other release tasks.

lib/mix/tasks/assets.ex
defmodule Mix.Tasks.Assets do
	use Mix.Task

	@shortdoc "Compiles CSS and JS assets"
	def run(_args) do
		# run sass command
		# run esbuild command
	end
end

We can now run this with mix assets, but it’s not doing anything quite yet. How can we get those watcher commands into this task, and run them as one-off processes?

One option is to simply copy the commands from config/dev.exs (without the zombie script), remove the --watch flags, and call them with Mix.shell.cmd/2. But it would be best if we don’t have to maintain the commands in two separate locations. Instead let’s create a new config file cmds.exs, which will contain a map of our commands without the --watch flag. Since this is not a file that needs to be compiled, we give it an .exs extension rather than .ex.

config/cmds.exs
%{
	zombie: Path.expand("../zombie.sh", __DIR__),
	sass: [
		"sass",
		"assets/css/app.scss",
		"priv/static/css/app.css"
	],
	esbuild: [
		"esbuild",
		"--bundle",
		"--sourcemap",
		"--target=es2016",
		"--outfile=priv/static/js/app.js",
		"assets/js/app.js"
	]
}

Back in config/dev.exs, we can use Code.eval_file/2 to pull that map in and use the commands in the watcher keyword list (while adding watch flags to both of them).

config/dev.exs
{cmds, _} = Code.eval_file("config/cmds.exs")

config :foobar, FoobarWeb.Endpoint,
	http: [port: 4000],
	debug_errors: true,
	code_reloader: true,
	check_origin: false,
	watchers: [
		{cmds.zombie, cmds.sass ++ ["--watch"]},
		{cmds.zombie, cmds.esbuild ++ ["--watch"]}
	]

Eval the same file in our mix task to run the same commands with different arguments for minification.

lib/mix/tasks/assets.ex
defmodule Mix.Tasks.Assets do
	use Mix.Task

	@shortdoc "Compiles CSS and JS assets"
	def run(_args) do
		{cmds, _} = Code.eval_file("config/cmds.exs")

		run_cmd(cmds.sass, ["--style=compressed"])
		run_cmd(cmds.esbuild, ["--minify"])
	end

	defp run_cmd(cmd, args) do
		cmd
		|> Enum.concat(if Mix.env() == :prod, do: args, else: [])
		|> Enum.join(" ")
		|> Mix.shell().cmd()
	end
end

Now if you run mix assets, it will produce the same priv/static/css and priv/static/js assets as the watcher scripts, but in one-off processes with minified files in production.

Compiling & Releasing

The last step is to integrate this custom task with your deployment (whether you’re using releases or just compiling for production). If our project has any static assets, we need to compile those assets prior to releasing with mix phx.digest (don’t run this yet!) This will create cached versions of every file in priv/static, along with a cache manifest file for optimized serving in production.

The problem is since we’re saving source asset files (excluding CSS and JS) in priv/static rather than copying them over from assets/, running the digest command will muddle up that folder with a ton of digested files that we don’t want to mingle with our originals.

Instead, let’s create an alias in mix.exs for digesting our static files to a different location while making sure the custom assets task runs first.

mix.exs
defp aliases do
	[
		setup: ["deps.get"],
		"phx.digest": ["assets", "phx.digest -o priv/dist"]
	]
end

Since priv/dist will be a folder of generated files, let’s add it to the gitignore as well.

.gitignore
# Since we are building assets from assets/,
# we ignore priv/static. You may want to comment
# this depending on your deployment strategy.
# /priv/static/
/priv/static/css/
/priv/static/js/
/priv/dist/

Go ahead and run mix phx.digest now, and take a look at the digested files in priv/dist.

Since we’re now using a different static folder in production, we need to tell our application to look in priv/dist for the cache manifest rather than the default of priv/static.

Update the cache_static_manifest path in config/prod.exs, and the from path in the Static plug of lib/foobar_web/endpoint.ex.

config/prod.exs
config :foobar, FoobarWeb.Endpoint,
	url: [host: "example.com", port: 80],
	cache_static_manifest: "priv/dist/cache_manifest.json"
lib/foobar_web/endpoint.ex
plug Plug.Static,
	at: "/",
	from: if(Mix.env() == :prod, do: {:foobar, "priv/dist"}, else: :foobar),
	gzip: false,
	only: ~w(css fonts images js favicon.ico robots.txt)

That’s it! You should now be able to compile for production, along with having a speedy asset pipeline that generates minified CSS/JS and properly digested static files.

# required environment variable for running in prod
$ export SECRET_KEY_BASE=$(mix phx.gen.secret)

$ mix deps.get --only prod
$ MIX_ENV=prod mix compile
$ MIX_ENV=prod mix phx.digest
$ MIX_ENV=prod mix phx.server

The source code for this project can be found on GitHub.

Update: Phoenix 1.6 now ships with ESBuild by default 🎉