1. Introduction

In this section we will look at what design tokens are and explain key design token concepts:

• What a design token and design token file can look like
• What transforms and formats are and how they turn design tokens into source code
• How popular design token frameworks work and their similarities

1.1 Checkout the package

All of the code and steps for this workshop are available on the github repository in branches. This workshop is a live-coding workshop, so we will explore and learn about concepts by building things in code!

Let's start by cloning the workshop repository to the step-1 branch:

git clone -b step-1 https://github.com/dbanksdesign/clarity-2020-design-token-workshop.git

Then lets go into the newly created directory with cd:

cd clarity-2020-design-token-workshop

Open the repository in whatever text editor or IDE you feel comfortable with. I am quite fond of VSCode myself. You can open it in VSCode with code .. There is not much to see there yet, it is an empty repository with a README and gitignore file. But there are branches with a more complete package. If you would like to fork this repository instead so you can commit changes, that is cool too!

First, let's initialize this repository as a NPM package using NPM (or Yarn whichever you prefer).

npm init

It will ask you some questions, for this workshop I will use this information:

• name: clarity-design-tokens
• version: leave blank (this chooses the default)
• description: leave blank
• entry point: leave blank
• test command: leave blank
• git repository: leave blank
• keywords: leave blank
• author: put your name
• license: leave blank

Before we start, please add:

"private": true

To the package.json file that was just created so we don't accidentally publish an NPM package with this name 😬.

1.2 Create a design token file

Design tokens at the most fundamental level are structured data in the form of key-value pairs. Similar to CSS or SCSS variables which have a name, $colorBackgroundPrimary and value #fff. This highlights the point that design tokens as a concept can be CSS variables, if you are using them to codify design decisions in a structured way.

The reason design tokens are often defined in a static data format like JSON or YAML is that these languages are fairly universally understood in programming languages. Additionally, it is often useful to store extra information, or metadata, about each token like a description.

In this workshop we will be building the technical version of design tokens: defined in a platform-agnostic way so that they can be used on different platforms to have a cohesive experience across platforms.

What is the most simple and universal design token?

{
  "colorBackgroundPrimary": "#fff"
}

One interesting thing about design tokens is you can include extra information, metadata, about them.

{
  "colorBackgroundPrimary": {
    "value": "#fff",
    "type": "color",
    "comment": "The primary background color"
  }
}

colorBackgroundPrimary:
  value: "#fff"
  type: color
  comment: The primary background color

Generally we can think of design tokens as less of a key-value pair, but a named object that has certain properties like a value. Having a design token as an object rather than a primitive like a string allows us to add extra information, which will be useful later. Both Theo and Style Dictionary use value to denote the value... of the design token for lack of a better word.

We are going to start with a simple design tokens file written in JSON. Create a new file in the root of the repository called tokens.json and let's add some design tokens to it.

{
  "size-padding-small": { "value": 0.5 },
  "size-padding-medium": { "value": 1 },
  "size-padding-large": { "value": 1.5 }
}

🎉 Yay! We created our first design tokens! 🎉

But our journey doesn't stop here. We need to be able to use these design tokens in our design systems and applications...

1.3 Create a build tool

Adding some color definitions in a JSON file is fine, but we need to use it in our applications. For design tokens to be used in real applications they need to be converted something the platforms (Android, iOS, Web, etc.) and languages (Kotlin, Swift, CSS, etc.) can understand. To do that requires 2 steps: transformation and formatting. To start we will create a simple build tool will make those 2 concepts clear.

Transforms take each design token and transform their name, value, or metadata so that they can be understood in a specific language or platform. Let's take a color for instance. In CSS you can define a color in many ways: hex, HSL, or RGB. But not every platform or language understands those, Android only understands hex codes natively and iOS uses a UIColor class that has red, green, and blue channels from 0-1 (instead of 0-255 like in CSS). Also the names of the tokens can depend on the platform or language too. For example on iOS and Android you can't use dashes in names, and your Android and iOS apps might have different naming conventions like Pascal case or snake case.

Formats take transformed design tokens and create files with them. Now that the names and values of the design tokens are in formats

Really what we are doing is taking some structured static data and transpiling it into source code that can then be used.

To create a build script, or transpiler if you will, let's use Node. Create a file called build.js. Now import your design tokens file:

const tokens = require('./tokens.json');
console.log(tokens);

Now let's add an NPM script to run this file in our package.json file. Add this in the "scripts" object just above the "test" command:

"build": "node build.js",

Now we can run npm run build that will run that Node file and you should see this output:

> node build.js
{
  'size-padding-small': { value: 0.5 },
  'size-padding-medium': { value: 1 },
  'size-padding-large': { value: 1.5 }
}

1.4 Transform the design tokens

For design tokens to be understood on different platforms and in different languages their names and values need to be in the right shape. So let's take our token object and transform each token so that it can be used in Sass (SCSS).

To do that we will take our design token object and turn it into a flat array. Each design token object will get a name and value attribute. For the name we prepend it with a $ because that is what SCSS variables use. The value we will add the rem unit because we defined the design tokens as unitless.

const tokens = require('./tokens.json');
const transformedTokens = Object.keys(tokens).map(key => {
  return {
    name: `$${key}`,
    value: `${tokens[key].value}rem`
  }
});
console.log(transformedTokens);

Let's run the npm run build command again and now we see a list of transformed tokens:

> node build.js
[
  { name: '$size-padding-small', value: '0.5rem' },
  { name: '$size-padding-medium', value: '1rem' },
  { name: '$size-padding-large', value: '1.5rem' }
]

1.5 Format the design tokens

Now that the design tokens have been transformed we need to take them all and format them into a source code file, like a CSS file. Because we need to create a file, we will be turning our token object into a string. Let's take the

const tokens = require('./tokens');
const transformedTokens = Object.keys(tokens).map(key => {
  return {
    name: `$${key}`,
    value: `${tokens[key].value}rem`
  }
});
const formattedTokens = transformedTokens.map((token) => {
  return `${token.name}: ${token.value};`
}).join('\n');
console.log(formattedTokens);

Run the npm run build command and you should see:

> node build.js
$size-padding-small: 0.5rem;
$size-padding-medium: 1rem;
$size-padding-large: 1.5rem;

Now that we have a string of the CSS code we want to output, we can write it to a file. First let's make a dist/ directory to put our generated files in. In a terminal write:

mkdir dist

const fs = require('fs');
const tokens = require('./tokens');
const transformedTokens = Object.keys(tokens).map(key => {
  return {
    name: `$${key}`,
    value: `${tokens[key].value}rem`
  }
});
const formattedTokens = transformedTokens.map((token) => {
  return `${token.name}: ${token.value};`
}).join('\n');
fs.writeFileSync(`dist/variables.scss`, formattedTokens);

Now lets run our build script one more time 🤣 and see our generated SCSS file!

node build.js

🎉 Yay! Our design tokens can be used now! 🎉

1.6 Make a change

Let's see how changing our design tokens will affect the output. Maybe 3 spacing sizes is not enough, we want an XL size now. Before we do that, I don't know about you, but I'm getting tired of having to run that command over and over. Let's set up a simple script to watch for changes and re-run the build command. Start by adding chokidar-cli as a dev dependency

npm install chokidar-cli --save-dev

Now we can add a script in our package.json file:

"watch": "chokidar \"tokens.json\" -c \"npm run build\"",

Now run npm run watch and it will keep the process open and wait for changes to our token file.

Add an xl spacing design token to the tokens.json file and save it:

{
  "size-padding-small": { "value": 0.5 },
  "size-padding-medium": { "value": 1 },
  "size-padding-large": { "value": 1.5 },
  "size-padding-xl": { "value": 3 }
}

Take a look at dist/variables.scss and we can see our new design token in there! 🎉

But design tokens are more than just padding. Let's add some colors to our tokens.json file:

{
  "size-padding-small": { "value": 0.5 },
  "size-padding-medium": { "value": 1 },
  "size-padding-large": { "value": 1.5 },
  "size-padding-xl": { "value": 3 },
  "color-background-primary": { "value": "#fff" },
  "color-background-secondary": { "value": "#eee" }
}

Can you guess what will happen?

$spacing-small: 0.5rem;
$spacing-medium: 1rem;
$spacing-large: 1.5rem;
$spacing-xl: 3rem;
$color-background-primary: #fffrem;
$color-background-secondary: #eeerem;

That's not going to work! 😫

Also if we want to use these tokens on other platforms we need to do the transforms and formats again! This is where frameworks like Theo, Style Dictionary, and Diez come in. They help do some of the heavy lifting so that you don't need to worry about a lot of things. You could stop here and build a super customized build system to generate files from your design tokens if you wanted.

1.7 Set up Style Dictionary

First, stop the watch process by press ctrl+c on a Mac (I don't know the equivalent on PC...)

For the rest of this workshop we will be working with Style Dictionary just so we only do things once, but all of the lessons and concepts can be applied to any framework or if you decide to build your own tool.

npm install style-dictionary --save-dev

Create a file called sd.config.js (the name actually doesn't matter). This is where we will put the configuration for Style Dictionary. Style Dictionary uses either a JSON file for configuration, or a node module, or you can use a node file itself and call Style Dictionary directly. For now we will use a node module. Copy and paste this code into sd.config.js

module.exports = {
  // source tells style dictionary where to find the design token files
  // it accepts an array of paths that can be globs
  source: ["tokens.json"],
  // platforms is an object where we define what platforms we want to target
  // the name of the platform doesn't matter, it is mostly just so you can see in the output
  // which platform is being built. 
  platforms: {
    web: {
      // Each platform defines either a transformGroup an array of transforms. If you
      // are using Theo it calls a transformGroup a transform and a transform is a valueTransform
      // We will get into transforms a bit more in the next section so don't worry
      transformGroup: "web",
      // Next we can optionally define a path to build all the files for this platform
      // for now we will use `dist`
      buildPath: "dist/",
      // Finally an array of files to build, you can build as many files as you need
      files: [{
        // Each file has a format, this is a reference to a built-in format
        format: "scss/variables",
        // And a destination tells where to write the file, this is appened to the buildPath
        destination: "variables.scss"
      }]
    }
  }
}

Let's change our tokens.json file a bit. Instead of a flat structure, let's make it nested.

{
  "size": {
    "padding": {
      "small": { "value": 0.5 },
      "medium": { "value": 1 },
      "large": { "value": 1.5 },
      "xl": { "value": 3 }
    }
  },
  "color": {
    "background": {
      "primary": { "value": "#fff" },
      "secondary": { "value": "#eee" }
    }
  }
}

Now update the build script in our package.json file to use style dictionary:

"build": "style-dictionary build --config ./sd.config.js"

Now npm run build will run the style dictionary build command, which accepts a configuration file, the file we just wrote. The style dictionary build command works roughly like this:

1. Find all the design token source files
2. Merge them together
3. Iterate over each platform defined in the config and
   1. Perform the transforms defined in the transformGroup or transforms array
   2. Build each file in the files array using the format specified

If you run npm run build in the terminal you should see this output:

> style-dictionary build --config ./sd.config.js
web
✔︎  dist/variables.scss

And if you take a peek at the dist/variables.scss file you should see:

// Do not edit directly
// Generated on Mon, 31 Aug 2020 23:47:49 GMT
$size-padding-small: 0.5rem;
$size-padding-medium: 1rem;
$size-padding-large: 1.5rem;
$size-padding-xl: 3rem;
$color-background-primary: #ffffff;
$color-background-secondary: #eeeeee;

1.8 Other frameworks

Before we move on, let's take a look at some frameworks and see how they work.

Theo

Theo is the first design token framework released by Salesforce. Theo and Style Dictionary are very similar and even share some concepts like transforms and formats!

The main difference of Theo is that it does not merge token files together. Instead you pass it a single toke file (you can import)

# buttons.yml
props:
  button_background:
    value: "{!primary_color}"
imports:
  - ./aliases.yml
global:
  type: color
  category: buttons

# aliases.yml
aliases:
  primary_color:
    value: "#0070d2"

const theo = require("theo");
theo
  .convert({
    transform: {
      type: "web",
      file: "buttons.yml"
    },
    format: {
      type: "scss"
    }
  })
  .then(scss => {
    // $button-background: rgb(0, 112, 210);
  })
  .catch(error => console.log(`Something went wrong: ${error}`));

Diez

Diez is a newer framework built by Haiku. It offers more of an ready-to-use solution. You can think of Style Dictionary and Theo like shopping at a grocery store and Diez like getting a meal-kit delivery service. You can make a delicious meal with both, one requires a bit more work but is more completely customizable, and the other is more consistent.

Choose whichever framework or custom solution fits your needs and makes the most sense to your organization. As long as you are using design tokens, however you implement them, I am happy 😁

Hopefully you now have an understanding of what design tokens are and how they can be implemented using a framework or a custom build tool. Don't worry if our design tokens seem a bit basic right now. Our journey is far from over...