Working around Vercel’s 4KB Environment Variables Limit for Node.js

Vercel’s 4KB environment variable limit stems from using AWS Lambda as their infrastructure provider. While Vercel has a documented solution for working around this limitation, it came with the drawback of large application code changes due to not using environment variables for secrets anymore plus the introduction of a new API endpoint.

Without a quick and easy way to implement an alternative that still allows environment variables to be used for secrets, the risk is teams resorting to insecure practices such as bundling secrets unencrypted in .env files.

At Doppler, we’re able to individually help customers who’ve hit this limitation using our Vercel integration, but we also didn’t have an off the shelf option.

So we set out to build an open source, secure Node.js solution that would allow secrets of any size to be supplied as environment variables, compatible with every serverless platform, and that any developer or team could use, not just our customers.

The result, is the gitops-secrets package for Node.js and in this article, you’ll learn how to bring a GitOps Secrets workflow to your Vercel applications and make environment variable limits a thing of the past.

Vercel GitOps Secrets Workflow

A Vercel GitOps Secrets workflow is a three step process in every environment:

1. Fetch secrets from a secrets manager or other secrets store
2. Bundle encrypted secrets into the build
3. Decrypt secrets within the application runtime.

No additional CLIs or external tooling is required for secrets encryption and decryption with all APIs provided by Node’s crypto package.

Take a look at the Vercel GitOps Secrets Next.js repository if you’re interested in a working example you can instantly deploy to Vercel.

Otherwise, let’s dive into bringing this workflow to an existing application but if you’d prefer to jump straight to a working example, check out the vercel-gitops-secrets-nextjs.

---

Start by installing the gitops-secrets package:
npm install gitops-secrets
Then if you haven’t already, install the Vercel CLI
npm i -g vercel
Then link the application codebase on your local machine with your Vercel application by changing into the root directory of your application by running:
vercel link

 

Master Encryption Keys

Create a unique and cryptographically random GITOPS_SECRETS_MASTER_KEY for every environment in Vercel by running:

node -e 'process.stdout.write(require("crypto").randomBytes(16).toString("hex"))' | vercel env add GITOPS_SECRETS_MASTER_KEY development
node -e 'process.stdout.write(require("crypto").randomBytes(16).toString("hex"))' | vercel env add GITOPS_SECRETS_MASTER_KEY preview
node -e 'process.stdout.write(require("crypto").randomBytes(16).toString("hex"))' | vercel env add GITOPS_SECRETS_MASTER_KEY production

Alternatively, you can create each GITOPS_SECRETS_MASTER_KEY variable from the Vercel dashboard.

Secrets Provider

A secrets provider is responsible for fetching your secrets from a secrets manager or encrypted storage backend in JSON format. In most cases, it should be as simple as a single async function that fetches secrets using an SDK or HTTP API.

Check out the built-in Doppler provider for a real example, but a valid provider could be as simple as encrypting a JSON payload from an environment variable injected from a GitHub Secret:

const secrets = require("gitops-secrets");

const payload = JSON.parse(process.env.GITHUB_JSON_SECRET)
secrets.build(payload, { path: "lib/secrets.js" }); 

Doppler Secrets Provider

So you can follow along, we’ll use Doppler as the secrets provider but the steps will be similar for any other secrets manager.

Create your Doppler account by installing the Doppler application from Vercel’s marketplace or use Doppler’s signup page.

On your local machine, install the Doppler CLI then authenticate by running:

doppler login

Change into the root of your Vercel application folder and create a Doppler project of the same name by running:

PROJECT"${PWD##*/}"
doppler projects create $PROJECT

Configure the default environment to be Development:

doppler setup ---project $PROJECT --config dev

Then rename Doppler’s default Staging environment to Preview for consistency with Vercel:

doppler environments rename --project $PROJECT stg --name Preview --slug prev -y

Next, you can import your secrets to Doppler using the dashboard or upload them using the CLI if there are in ENV, JSON, or YAML format:

doppler import --config dev dev-secrets.json
doppler import --config prev prev-secrets.json
doppler import --config prd prod-secrets.json

With your secrets imported into Doppler, each Vercel environment will need a Doppler Service Token named “DOPPLER_TOKEN` which gives the Doppler provider read-only access to the secrets for a single environment.

We can automate this using the Doppler and Vercel CLI:

echo -n "$(doppler configs tokens create vercel-gitops --config dev --plain)" | vercel env add DOPPLER_TOKEN development
echo -n "$(doppler configs tokens create vercel-gitops --config prev --plain)" | vercel env add DOPPLER_TOKEN preview
echo -n "$(doppler configs tokens create vercel-gitops --config prd --plain)" | vercel env add DOPPLER_TOKEN production

Alternatively, you can use the Doppler and Vercel dashboard to create the Service Token for each environment manually.

You’ll need a new script for bundling the encrypted secrets into your build which can be generated in one of two formats:

Because Vercel’s File System API is more geared towards serving static resources than it is reading them, choosing the JS file is the best and easiest way to get started.

We’ll place the script inside a bin directory and save the generated JS file into a lib directory or similar to provide a location accessible to any code in your application.

Create the bin/encrypt-secrets.js file:

const secrets = require("gitops-secrets");

async function main() {
const payload = await secrets.providers.doppler.fetch();
secrets.build(payload, { path: “lib/secrets.js” });
}

main();

The generated lib/secrets.js is then imported like any other JS file in your application and we’ll cover this in more detail in the Decryption section.

Now add an encrypt-secrets script to your package.json:

"scripts": {
...
"encrypt-secrets": "node ./bin/encrypt-secrets.js"
}

You can quickly test the encrypt-secrets script locally using a throwaway DOPLER_TOKEN and GITOPS_SECRETS_MASTER_KEY environment variables by running:

GITOPS_SECRETS_MASTER_KEY="$(node -e 'process.stdout.write(require("crypto").randomBytes(16).toString("hex"))')"
DOPPLER_TOKEN="$(doppler configs tokens create temp --max-age 1m --plain)"
npm run encrypt-secrets

Vercel Settings

Update your Vercel project settings in the dashboard so npm run encrypt-secrets is run prior and in addition to your existing build and development commands:

This enables your GitOps Secrets flow to work locally as well.

Decryption

Now that all the pieces are now in place, let’s cover decrypting and accessing secrets at runtime.

Decryption works because theGITOPS_SECRETS_MASTER_KEY environment variable used during the build process also exists and is the same at runtime.

Because decryption relies on environment variables, this prevents secrets from accidentally being exposed in frontend code as environment variables are only accessible in API endpoints, getServerSideProps and getStaticProps.

The generated JS file at lib/secrets.js doesn’t just contain the encrypted cipher text but also comes with convenience methods for accessing secret values.

The examples below use CommonJS syntax but ES modules are also fully supported as the module will be generated in the module format used by your application.

For accessing secrets, you could use populateEnv to emulate how Vercel injects secrets as environment variables:

const { loadSecrets } = require("../lib/secrets.js")
loadSecrets().populateEnv();

process.env.API_KEY 

Or simply access them directly:

const { loadSecrets } = require("../lib/secrets.js")
const secrets = loadSecrets();


secrets. API_KEY

It works equally well from anywhere in your Vercel application, including within pages provided loadSecrets is called inside getServerSideProps or getStaticProps functions:

const { loadSecrets } = require('../lib/secrets')


export async function getServerSideProps() {
const secrets = loadSecrets()
}

 

Local Development

A GitOps secrets workflow will only be successful if the development experience doesn’t suffer as a result.

And thanks to the Vercel CLI, all you need for running locally is:

vercel dev

Trigger Production Deploys on Secret Change with Doppler Webhooks

You configure Doppler to trigger a production redeploy in Vercel when secrets in the Production environment change using webhooks.

In the Vercel dashboard, create a Production deploy webhook by navigating to Settings > Git, then click the Create Hook button.

Copy the webhook URL, then open your project in the Doppler dashboard and click Webhooks from left menu. Click the + Add button, pasting in the webhook URL and once the webhook has been created, check the PRD checkbox.

To test, simply change a secret in the Doppler Production environment and you should observe a Production deploy in progress in Vercel.

Awesome work!

Now you know how to work around Vercel’s 4KB environment variable limit using a GitOps Secrets workflow.

Take a look at the Vercel GitOps Secrets Next.js repository to see a full working example you can deploy to Vercel.

The gitops-secrets package is currently in developer preview and we would love to know what you think!

Join us in our Doppler community forum, find us on Twitter, and for bugs or feature requests, create an issue in the open source repository.

Thanks for reading!