Modern Web Development: where you need 500 packages to build Bootstrap

Attempt 0: no JS stuff, no node_modules

I installed a Sass compiler. There are web services like cssminifier.com that can be easily used with curl in a Bash script. Autoprefixer has a webpage that lets you use the service without installing it as well. The catch is, the code runs locally in your web browser. Automating a web browser requires some effort. I decided to leave this part un-automated. Here is the Bash script I hacked together (with some messages removed):

sass bootstrap-kw.scss > bootstrap.noprefix.css
sass bootstrap-kw-dark.scss > bootstrap-dark.noprefix.css

echo "Go to https://autoprefixer.github.io/."

pbcopy < bootstrap.noprefix.css
echo -n "(light) Paste the clipboard contents and copy the output, then press Enter."
read temp
pbpaste > assets/css/bootstrap.css

sleep 1

pbcopy < bootstrap-dark.noprefix.css
echo -n "( dark) Paste the clipboard contents and copy the output, then press Enter."
read temp
pbpaste > assets/css/bootstrap-dark.css

curl -X POST -s --data-urlencode 'input@assets/css/bootstrap.css' https://cssminifier.com/raw > assets/css/bootstrap.min.css
curl -X POST -s --data-urlencode 'input@assets/css/bootstrap-dark.css' https://cssminifier.com/raw > assets/css/bootstrap-dark.min.css

The “manual copy” solution was inconvenient, but it worked.

Well, most of the time Some lags/glitches with the clipboard meant that sometimes, files had the incorrect content. So, I wanted to fix it, and build it in a more modern, JS-y way. The way Bootstrap does it is a lot of shell commands (that run various Node tools). I don’t feel like building this pipeline with Bash, it would feel fragile. Let’s do it the JS way.

Attempt 1: webpack

I’ve used webpack for another project of mine. It was okay, and it did the job (namely, compiling TypeScript into browser-usable JS).

I wanted to give it a try for this one. I googled “webpack sass”. The first result was sass-loader. The pipeline for it was:

"style-loader", // creates style nodes from JS strings
"css-loader", // translates CSS into CommonJS
"sass-loader" // compiles Sass to CSS, using Node Sass by default

Let’s recap. Someone thought that the right way to do CSS is to use JS imports.

Yes. import "./style.css"; in a JS file. So that your fancy build tool knows about CSS.

Webpack wasn’t the right tool for my project, but even if I had JS code there, WHY WOULD I MENTION STYLESHEETS IN MY JS CODE?! Webpack’s website also lists .jpg and .png assets, are they meant to be imported in JS as well? This is absurd.

Going back to googling “webpack sass”… The next two results were Medium posts. The stupidity of Medium as a blog platform notwithstanding, one of the posts was from 2017, referring to webpack 2. The next post was a year older, a completely unreadable mess, and it was for webpack 4. That’s not helpful in any way.

Attempt 2: Gulp

Let’s try something else from the JS world: Gulp. Now, the tool is not terrible, but it still requires a lot of dependencies.

The pipeline that was required for this task sounds very simple:

bootstrap-kw{,-dark}.sass → Sass compiler → Autoprefixer → bootstrap{,-dark}.css → minify → bootstrap{,-dark}.min.css

The Gulp version is fairly simple: (I based it on examples on Gulp’s website, and pages of all my dependencies).

const { src, dest } = require('gulp');
const minifyCSS = require('gulp-csso');
const sass = require('gulp-sass');
const postcss = require('gulp-postcss');
const autoprefixer = require('autoprefixer');
const rename = require("gulp-rename");
sass.compiler = require('node-sass');


function css() {
    return src('*.scss')
        .pipe(sass.sync().on('error', sass.logError))
        .pipe(postcss([autoprefixer()]))
        .pipe(rename(function (path) {
            path.basename = path.basename.replace("-kw", "")
        }))
        .pipe(dest('assets/css'))
        .pipe(minifyCSS())
        .pipe(rename(function (path) {
            path.basename += ".min"
        }))
        .pipe(dest('assets/css'));
}

exports.default = css;

A node_modules extravaganza

Can you see all the require lines at the top? Every one of them is a dependency of my build script. With the exception of gulp-rename, which IMO should be a built-in part of Gulp (it’s 45 lines of code and no external dependencies), the list is sensible.

Well, I already mentioned the size of node_modules: 51 MiB according to du (size-on-disk measurement). How many packages are there?

545. Five hundred and forty-five packages.

Whoa, when did that happen? Most of it comes from gulp/gulp-cli (384 packages), with node-sass taking the second place (177 packages). Some of those are shared between libraries, and a few more belong to the other requirements. And many of these dependencies are a disgrace to programming.

After a full install of my package.json, npm says added 545 packages from 331 contributors and audited 10500 packages in 22.458s. I’ve implicitly agreed to licenses imposed by 331 random people. All to build some simple CSS files out of SASS.

Let’s go on a tour of node_modules and see what we ended up with.

// Copyright © Sindre Sorhus, MIT license
module.exports = Number.isNaN || function (x) {
    return x !== x;
};

/*!
 * is-even <https://github.com/jonschlinkert/is-even>
 *
 * Copyright (c) 2015, 2017, Jon Schlinkert.
 * Released under the MIT License.
 */

'use strict';

var isOdd = require('is-odd');

module.exports = function isEven(i) {
  return !isOdd(i);
};

/*!
 * is-odd <https://github.com/jonschlinkert/is-odd>
 *
 * Copyright (c) 2015-2017, Jon Schlinkert.
 * Released under the MIT License.
 */

'use strict';

const isNumber = require('is-number');

module.exports = function isOdd(value) {
  const n = Math.abs(value);
  if (!isNumber(n)) {
    throw new TypeError('expected a number');
  }
  if (!Number.isInteger(n)) {
    throw new Error('expected an integer');
  }
  if (!Number.isSafeInteger(n)) {
    throw new Error('value exceeds maximum safe integer');
  }
  return (n % 2) === 1;
};

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
3. All advertising materials mentioning features or use of this software
   must display the following acknowledgement:
     This product includes software developed by the University of
     California, Berkeley and its contributors.

├─ MIT: 380
├─ ISC: 64
├─ Apache-2.0: 10
├─ BSD-3-Clause: 10
├─ BSD-2-Clause: 3
├─ CC-BY-3.0: 2
├─ BSD-3-Clause OR MIT: 1
├─ MIT*: 1
├─ (MIT OR Apache-2.0): 1
├─ CC-BY-4.0: 1
├─ AFLv2.1,BSD: 1
├─ MPL-2.0: 1
├─ (BSD-2-Clause OR MIT OR Apache-2.0): 1
├─ CC0-1.0: 1
└─ Unlicense: 1

Attempt 2: back to Bash

I decided to get rid of Gulp, it’s not necessary for this pipeline. I replaced it with Bash and postcss-cli. node-sass was replaced by dart-sass (a two-file binary distribution), and csso was replaced by cssnano (it works with postcss). Here is the resulting Bash file:

sass bootstrap-kw.scss | npx postcss --no-map --use autoprefixer -o assets/css/bootstrap.css
sass bootstrap-kw-dark.scss | npx postcss --no-map --use autoprefixer -o assets/css/bootstrap-dark.css
npx postcss --no-map --use cssnano -o assets/css/bootstrap.min.css assets/css/bootstrap.css
npx postcss --no-map --use cssnano -o assets/css/bootstrap-dark.min.css assets/css/bootstrap-dark.css

The simplified dependency list cost me 37 MiB of disk space, and I’ve got 438 packages from 232 contributors.

Attempt 3: node CLIs are unnecessary

Let’s try something else: replace npx postcss with a custom tool.

const fs = require('fs');
const getStdin = require('get-stdin');

const postcss = require('postcss');
const autoprefixer = require('autoprefixer');
const cssnano = require('cssnano');

const name = process.argv[2];

getStdin().then(css => {
    postcss([autoprefixer]).process(css, {from: undefined}).then(result1 => {
        fs.writeFileSync(`assets/css/${name}.css`, result1.css);

        postcss([cssnano]).process(result1.css, {from: undefined}).then(result2 => {
            fs.writeFileSync(`assets/css/${name}.min.css`, result2.css)
        });
    });
});

The bash script now pipes sass output to node run_postcss.js bootstrap(-dark).

Doing this… cost me a new dependency. Its name is get-stdin. We’ve already met its author, Sindre Sorhus. While the library has its deficiencies (namely, it doesn’t support reading from TTY), it’s good enough. I could do it manually or use some other tricks, but since get-stdin does not pull in any other dependencies, I’m going to accept it. After cleaning up packages.json, we end up with:

$ npm install
added 144 packages from 119 contributors and audited 637 packages in 8.127s
found 0 vulnerabilities
$ du -hs node_modules
 21M    node_modules

Conclusion

The task at hand was very simple. So was the JS code (Gulp and custom) I had to write to implement it. But underneath, there was a mess of unknown, unaudited code, duplicated libraries, and libraries created effectively to bump people’s npm download stats. There were already incidents, like left-pad (the removal of which broke Babel), or event-stream (which was taken over and modified to steal cryptocurrencies). The modern web development ecosystem is a huge mess of dependencies and one-line packages. Some of them are necessary due to the lackluster JS standard library — but some are just useless. And some of these micro-packages would be better off as larger libraries.

Sure, the package count went down from the original 545 to 144. But the original point still stands: too much useless stuff.

PS. Five of the packages (in the “large” set) had a .DS_Store file left over. I’m wondering if there are any other files that shouldn’t be shipped with packages, lurking in node_modules directories all over the world…

PPS. I’ve replaced Disqus with Isso, because it had too many advertisements. If you experience any issues with the comment system (after force-refreshing), e-mail me.