Blog

Find out the news

Release: Yarn 4.0 🪄⚗️

Today is the day! After more than a year of work, our team is excited to finally put a fancy “stable” sticker on the first release from the 4.x release line! To celebrate, let’s make together a tour of the major changes; should you look for a more itemized list, take a look at the changelog.

Breaking Changes

Here’s what you need to know when upgrading from 3.x projects:

• We now require Node.js 18+.
• New projects created with yarn init won’t enable Zero-Install by default anymore.
• New projects created with yarn init will use Corepack rather than yarnPath.
• All official plugins (typescript, interactive-tools, …) are now included by default.
• The yarn workspaces foreach command has a slightly altered syntax.

Installing Yarn

Ever since the 2.0 our recommendation has been to install Yarn on a per-project basis using the yarnPath setting (automatically set either of yarn init -2 and yarn set version). We intentionally don’t release modern releases on the npm yarn package, so as not to break older projects which didn’t migrate yet.

To that end we used to recommend using the yarnPath setting pointing to a checked-in binary, but this pattern increased friction more than we liked - many people didn’t like the idea of adding a binary to their repository, however small. We listened, and worked conjointely with Node.js on a project called Corepack. Corepack is a tool shipped with Node.js 16+ that will automatically select the right package manager version to run depending on the project you’re working on.

Now that Corepack is shipped with both Node 18 and 20 we no longer need to rely on yarnPath, and as a result we updated our installation guide to reflect that. The yarn init -2 and yarn set version commands have been updated to favor updating the packageManager field when possible.

Note

Corepack knows which package manager version to use thanks to the standard packageManager field in your package.json. This field will typically be set via one of yarn init -2, yarn set version x.y.z, or the more generic corepack use yarn@x.y.z.

Hardened Mode

Yarn attempts to protect you from common attacks, and this is pushed even further by the introduction of the Hardened Mode. When operating under this mode, Yarn will perform two extra validations:

• It will validate the resolutions stored in the lockfile are consistent with what the ranges could resolve to.
• It will validate that the package metadata stored in the lockfile are consistent the remote registry metadata.

Together, those checks will prevent any attacker from surreptitiously modifying your lockfiles when making PRs to your project using Yarn (https://snyk.io/blog/why-npm-lockfiles-can-be-a-security-blindspot-for-injecting-malicious-modules/).

Tip

The Hardened Mode is enabled by toggling on enableHardenedMode, but it’s also automatically enabled when Yarn detects that it runs within a GitHub pull request on a public repository. This can be disabled by explicitly toggling off enableHardenedMode in your yarnrc file.

Caution

Installs operating under Hardened Mode constraints are significantly slower than usual as they need to perform many network requests that would be skipped otherwise. We don’t recommend enabling it by default - if you need it in a specific CI job, toggle it on via an environment variable:

export YARN_ENABLE_HARDENED_MODE=1

JavaScript Constraints

Yarn is the only package manager to implement a constraints engine. If you don’t know it, this feature lets you define a set of rules that your project must satisfy. For instance, the Yarn repository enforces that no two workspaces depend on different versions of any given dependencies, unless explicitly allowed.

Our constraints engine used to be powered by Tau-Prolog, a JavaScript Prolog implementation. Unlike imperative languages like JavaScript, Prolog uses a different model called logic programming - you define that something exists if a rule is true. It’s a very interesting pattern that integrates well with the concept of rule-based linting. Unfortunately, Prolog proved very complex to use, increasing the learning curve of constraints past the threshold we were comfortable with.

As a result, Prolog constraints are deprecated starting from Yarn 4, and they have been superseded by a shiny new JavaScript-based engine, with optional TypeScript support! We have been writing our own rules at Datadog with this framework for a couple of months now, with great success. You can also check the public Yarn repository for a practical example of the kind of rules you can enforce at the repository level, and the newly revamped documentation is there to help you quickly get up to speed.

Tip

The new optional enableConstraintsChecks setting will make Yarn run your constraints as part of yarn install. It’s a handy way to surface errors before having to wait for the remote CI to raise them, and since the new engine is so fast, it has almost no impact on your install time 🚀

TypeScript Integration, Interactive Tools, …

Various features in Yarn used to be shipped as sideloaded plugins that needed to be managed separately from the main bundle. While this helped us build a plugin ecosystem, it also proved challenging to manage for our users. We implemented some features to make that easier (auto-upgrade plugins when you auto-update Yarn), but in the end the few KiBs we gained by not shipping all the features by default weren’t worth the confusion and friction they caused.

As a result, while Yarn still supports third-party plugins (and will continue to in the future), all the features and commands we build are now available as part of the main distribution. You can now use yarn upgrade-interactive and yarn stage without plugins and, if you have TypeScript configured in your project, Yarn will now auto-add and remove @types packages as needed whenever you update your dependencies with yarn add and yarn remove.

Improved User Interface

Various pieces of the UI got revamped to better convey information. For example, yarn install now tells you the packages you added, and their total weight. You will also notice it doesn’t print as much warnings around peer dependencies, as we now try to only print warnings for actionable situations:

// import addNextTerm from ”./release-4.0—add-next.term.dat”;

Another example is the yarn config command, which sports a new tree display and now also accepts an arbitrary number of settings as positional arguments, letting you select what you wish to see:

// import configTerm from ”./release-4.0—config.term.dat”;

Performances

The 4.0 isn’t lagging behind in performance improvements, and shows to be significantly faster at installs than the 3.6. For instance, here’s the difference in time to install Gatsby and its ~350MiB dependency tree from a cold cache. The 3x improved performances are due to a new package metadata cache which significantly improves performances of repeated installs:

hyperfine -L v stable,canary --prepare 'rm -rf ~/.yarn/berry/cache' 'cd $(mktemp -d) && yarn init -2 && yarn set version {v} && yarn && yarn add gatsby --mode=skip-build'

Benchmark 1: 3.6.0
  Time (mean ± σ):     65.599 s ±  2.214 s    [User: 82.952 s, System: 8.638 s]
  Range (min … max):   62.167 s … 68.277 s    10 runs

Benchmark 2: 4.0.0
  // highlight-next-line
  Time (mean ± σ):     16.724 s ±  0.928 s    [User: 14.622 s, System: 5.743 s]
  // highlight-next-line
  Range (min … max):   15.318 s … 18.110 s    10 runs

Summary
  4.0.0 ran 3.92 ± 0.25 times faster than 3.6.0

These changes make Yarn as fast as pnpm in most scenarios, although competition is still fierce 🔥

Fancy Website

As you probably noticed, our website received a major overhaul, both style and content! We worked on this new iteration for more than a year now, and we hope it’ll help you find better information, faster than before.

Some particular improvements:

• All referenced commands now link to their documentation (yarn install)
• All referenced options now have a tooltip explaining their goal (yarn --immutable-cache)
• Most pages were rewritten to be both simplified & clarified when needed
• The package page now shows various configurable checks, like whether a package supports CJS, ESM, has types, …

Our expertise lies in tooling more than building websites, so I’m sure various hanging fruits remain - especially around loading time. If you’re interested to help us, check the sources and please feel free to send PRs our way!

Closing Words

The journey to transition from Yarn 3 to Yarn 4 was a lengthy one, with a whopping 53 release candidates along the way, but we finally made it! Our aim for this new iteration has been to both decrease Yarn’s learning curve and improve your user experience, without the migration feeling overwhelming. We made concerted efforts to avoid making significant breaking changes unless we also had ways to automatically migrate projects, so if you encounter any issues that you believe the software should have addressed, share your feedback with us on Discord.

As for what lies ahead, it’s a bit too early to provide a definitive answer, but I can tell you I’m particularly intrigued by the potential for native Yarn builds. Performances has been under the spotlight lately, and I sometimes wonder how much overhead may have Node.js on the overall execution time. That being said, we don’t plan on undertaking another complete rewrite of the codebase, nor do we want to compromise the factors that make Yarn so contributor-friendly, so the specifics, as well as the timeline, are still under consideration.

In the meantime we’ll continue to build upon our existing foundations for the time being. From CLI completion and UI commands to reducing the learning curve and general upkeep, we have a broad array of improvements on our radar. So see you next time!

Maël Nison

Lead Yarn maintainer

October 23, 2023

8 min read

A word about Bun 💬

I’m sure many of you are curious about our position regarding Bun, the product from Oven, the company behind Bun (we’re going in cycles). It’s so fast, is there any merits to using Yarn?

First, we feel useful to point out that this sentence isn’t particularly new. We heard the same (often from the same people) asking why use Yarn when npm/pnpm/whateverpm have all its features or outspeed it. Answering that is a little tough, because the premise is wrong: other package managers don’t have its features¹², and the speed differences are at best marginal. They are a good fight, but we believe Yarn ultimately has a unique position that no other package managers emulates today.

Bun is an interesting case, though. It’s definitely much faster³. Can Yarn compete? We believe so.

First, remember today’s iteration of Yarn was developed over the course of two years by a team already experienced in package managers. Those tools are fickle beasts, and many edge cases loom around⁴. Bun may be fast, but is it correct? That’s something the community will have to figure out over time.

But stability isn’t everything: the feature set is an important facet of what makes a tool appealing. The developer experience (which includes the user interface) is another. The governance yet another. Yarn stills fits its niche: a complete tool that empowers its users, advocates for good practices, isn’t afraid to explore uncharted territories, and is protected from perverse corporate incentives.

With that said, I believe there’s a couple of things we can learn from Bun. Yarn was always intended to be distributed as a unique JS file for extreme portability across Node.js supported architectures. With Corepack now being the preferred install strategy, does it still matter? Should we experiment with native modules for future releases, that Corepack would transparently fetch as needed? Bun proved untapped performances could be exploited.

Of course it’s not just a matter of being native - Oven’s work follows interesting code patterns, and I’m curious how much of an impact they have on the resulting speed (at the cost of increased complexity, and making contributions harder).

I always fought against the idea that one package manager was enough for every single project out there, Yarn included. Our users are engineers: they have different requirements, different priorities, and different sensibilities. I found Yarn the appropriate tool for my projects, but I’m sure Zoltan is perfectly happy with pnpm and Microsoft with npm.

Will Bun reach some of your hearts? More than likely. Will it be a replacement? I can’t imagine that.

Footnotes

1. For example the portable shell, the constraints, the patching, … ↩

2. Or delegate to Yarn; did you know pnpm’s hoisted linker is literally using Yarn as a dependency? ↩

3. Although not as much as they pretend, which is a bit of a letdown. Marketing corrupts, eh? ↩

4. Here be dragons. ↩

Maël Nison

Lead Yarn maintainer

August 20, 2023

3 min read

Release: Yarn 3.2 🚢🔮

Welcome to the release notes for Yarn 3.2! This release is a little smaller than the 3.0 and 3.1, as we’ve hold off on some changes in preparation for our next major … but more on that later 😃

As always, keep in mind those are only the highlights, the full changelog is much more comprehensive.

Sponsoring

The Yarn org needs your help to make our work more sustainable! Please take a look at our OpenCollective and GitHub Sponsors pages for more details 😊

Libc Field

We implemented in 3.1 a feature we call “conditional dependencies”. The idea is simple: if a package is listed in the optionalDependencies field and its os / cpu fields don’t match the current machine, we don’t install them at all. This pattern is today used by various tools like Esbuild or SWC to avoid overfetching dependencies that systems wouldn’t needed.

One problem however is that while os and cpu are useful at differentiating systems, they aren’t the only parameters at play. In particular, knowing the standard C library against which native modules are built is critical: using a module linked against the glibc with a Node release built against musl would promptly crash.

To avoid this, Yarn now supports a libc array field in the package.json that currently accepts any of two values: glibc and musl. Just like os and cpu, packages will be skipped if they don’t match the host libc.

This isn’t the final iteration; while libc is a good improvement, more parameters could be taken into account. Both Yarn and npm have open proposals to address this situation, and we’ll see what we decide to implement.

New Command: yarn explain

It can be difficult to know how to react when facing errors. Our website tries to help with that by providing detailed explanations, but when you’re in your terminal this might not be the first thing you have in mind.

The new yarn explain command will let you get all the details about an error, right from your terminal:

In the future we’ll expand the documentation to cover more error messages, and may use yarn explain to aggregate some of the other similar mechanisms we already have (such as yarn explain peer-requirements).

UI Improvements

Every version we look for little UI annoyances to fix. This time is no exception with a couple of neat improvements:

• The resolution step will now have a spinning wheel; we can’t show a percent-based progress since we don’t know how many packages we’ll have to resolve until the end, but a spinner will at least let you know the process isn’t stuck.
  
  

• Errors thrown when cloning Git repositories were previously reported as regular stack traces. They will now have dedicated output.

Next Major

With 3.2 out of the door, we’ll now start working on the next major release: Yarn 4! We have an issue highlighting the things we currently have in mind, but generally speaking expect us to decrease the friction when starting new projects. Some highlights:

• We’ll drop support for Node 12, as it will reach EOL in April
  
  

• We’ll be exploring a new resolution algorithm that will prevent most of the attacks similar to the recent color.js hijacking.
  
  

• More commands will be integrated with Git; we used to refrain from doing so due to some related projects using Mercurial, but this isn’t the case anymore. Projects not using Git will still be able to use Yarn, but some features may not be available there.

  • To give you an idea of the kind of integration we have in mind, the yarn stage command (already available as a plugin) allows to automatically commit all dependency-related changes without impacting any other.
  
  
  

• The official plugins will be shipped by default, to reduce the friction. In practice the Yarn binary is very small, so we have some leeway to bundle everything together so that you don’t have to download more subparts.

  • Even if bundled by default they’ll technically remain plugins, so it doesn’t change anything for third-party plugin authors: our plugin API will remain a focus for us, and will keep improve.

And more! 🙂 We have plenty of other ideas to improve Yarn, so expect to see a significant amount of improvements in our next major - including lower friction when starting new projects or migrating older ones.

Maël Nison

Lead Yarn maintainer

February 21, 2022

4 min read

Release: Yarn 3.1 🎃👻

Welcome to the release notes for Yarn 3.1! We’re quite excited by this release, as it brings various improvements that we’ve all been looking forward to. Let’s dig into that!

As always, keep in mind those are only the highlights, the full changelog is much more comprehensive.

Sponsoring

The Yarn org needs your help to make our work more sustainable! Please take a look at our OpenCollective and GitHub Sponsors pages for more details 😊

Table of content

• Node.js Corepack Integration
• ESM Support
• New Install Mode: pnpm
• Conditional Packages
• Smart Changeset Filters
• New Workspace Syntax: workspace:^

Improvements

Node.js Corepack Integration

Did you know that Yarn now ships with Node? This is done via the Node.js Corepack project, which includes both the Yarn and pnpm binaries as shims. By adding the packageManager field to your package.json, you can enforce the use of a specific package manager & package manager version in a completely transparent way:

{
  "packageManager": "yarn@3.1.0"
}

Note that Corepack is available starting from Node.js 16.9, but is currently opt-in. Don’t forget to run corepack enable a single time to make sure the shims are globally installed!

We also improved in 3.1 the init command to properly support Corepack: running yarn init -2 will now automatically setup a Yarn Modern project, setting its packageManager field as required 💫

ESM Support

ESM has always been supported when using the node_modules linker, since it’s the same old install strategy that Node has always supported. However, with PnP taking ownership of the resolution pipeline, compatibility with ESM wasn’t a given and had to be implemented using the Loader Hook API.

While the Loader Hook API isn’t entirely stable yet, a large amount of work has been made lately and our team has been able to produce an initial experimental support for ESM modules. It should be enabled automatically if we detect that one of the packages in your dependency tree contains a "type": "module" field, but you can enable or disable it manually through your settings:

pnpEnableEsmLoader: true

Being experimental, it’s possible that some bugs may arise or that new Node releases bring some breaking changes around the API. Be sure to report issues on our bug tracker!

New Install Mode: pnpm

The pnpm package manager was one of the first tools to advocate for using symlinks when installing packages within the node_modules folder. While we went another way with PnP, we decided that the implementation cost was low enough that it would be worth adding support for this symlink-based install strategy as well.

Starting from Yarn 3.1, you can try out symlink-based installs by adding the following setting to your .yarnrc.yml file:

nodeLinker: pnpm

Conditional Packages

Esbuild and swc are two native packages that gained a lot of attention lately thanks to their impressive performances over their competitors. They recently revamped how their packages are built to avoid complex postinstall scripts, but did so in a way that was less efficient than before for Yarn projects.

Yarn 3.1 features a new optimization that kicks in when a package is listed as optionalDependencies and lists os and/or cpu fields. When that happens, Yarn will skip fetching and installing those packages unless they match the current system parameters.

In case you need to manually configure a strict set of package architectures to support (for example like in a zero-install case, where you want to read from an immutable set of packages), you can use the supportedArchitectures setting:

supportedArchitectures:
  os: [linux, darwin]
  cpu: [x64, arm64]

Smart Changeset Filters

The yarn workspaces foreach and yarn workspaces list commands now ships with brand new --since flags. When set, those commands will only execute against the packages that changed when compared to the main branch (either main or master, depending on the branches in your repository).

This can come in handy if you wish to only run builds in some specific workspaces, or just get a list of the workspaces which changed for scripting purposes:

yarn workspaces foreach --since run eslint .
yarn workspaces list --since

The --since flag also accepts an optional argument (--since=${commit-ish}) to manually define a source from which the changes should be derived.

New Workspace Syntax: workspace:^

Workspaces supported a special syntax via workspace:*, with those ranges being replaced at publish-time by exact ranges corresponding to the real version of the target workspace. However, if you wanted to use a caret instead of an exact range, you had to use the verbose workspace:^x.y.z form, which Yarn updated repo-wide after each publish.

Yarn now supports workspace:^ and workspace:~ as well, making it much easier to cross-reference workspaces within a monorepo where most packages are intended to be published, by preventing a good amount of the merge conflicts that used to happen after Yarn updated the verbose ranges.

Additionally, as a special case, this syntax is now allowed in the peerDependencies field as well:

{
  "peerDependencies": {
    "@my/other-package": "workspace:^"
  }
}

Maël Nison

Lead Yarn maintainer

September 25, 2021

4 min read

Release: Yarn 3.0 🚀🤖

Hello! Long time no see! Back in December, we decided to start working on our next major release, the 3.0. It took a bit of time to do everything we intended to do, but here we are! So let’s talk a bit about what it changes, and what it brings. Note that these are only the highlights, the full changelog is much more comprehensive.

Governance

Back when the project was started in 2017, we didn’t took the time to establish a formal governance document. This is now addressed, and our team composition can be found here. It doesn’t change anything in practice (this is how we worked for more than two years now), but we hope it may give you a better understanding as to how we work and reach consensus.

OpenCollective

From 2017 to 2019 Yarn was mostly maintained by Facebook engineers. While it worked relatively well, the Yarn 1 -> 2 release also proved to be the right time to expand our active team to other horizons, and nowadays no two of our active contributors work at the same company - and none at Facebook.

Consequently, we’ve decided to setup an OpenCollective (or GitHub Sponsors) to give our supporters a way to both express their thanks to our team, and give us resources we can then inject back into the project.

Breaking Changes

While the migration from Yarn 1 to Yarn 2 brought some discomfort, the migration to Yarn 3 should prove easier - regardless of the version you come from. The user-facing breaking changes we made this season are mostly little details that may only affect you in very specific cases:

• Node 10 isn’t supported anymore
• Plug’n’Play hooks are now called .pnp.cjs (vs .pnp.js)
• Virtual folders are now called __virtual__ (vs $$virtual)
• The editor SDKs have been moved to @yarnpkg/sdks
• Etc; full list here

Even for Yarn 1 users, migrating from 1 to 3 should be easier: we made it so that Yarn will detect when this situation arises to then automatically enable the node-modules linker. That alone should address most of the problems you may have been hitting when attempting the upgrade - and for everything else, make sure to take a look at our Migration Guide which got significantly improved over the past year.

Support for the exports field

When using Yarn 3 w/ PnP, the exports field will be properly resolved regardless of your Node version. If you’re not familiar with this field, you can see it as a way to:

• Replace the main field
• Soft-prevent accessing arbitrary files in the package
• Conditionally remap files depending on the context (bundlers, …)

Performances

Various tweaks have been made to address some of the largest resource consumptions in Yarn. Installs have been improved (turning us faster than pnpm in some scenarios, which is quite a feat!), but not only: script execution tends to have a natural overhead, but bugs in 2.4 and prior caused this overhead to grow relative to the size of the project itself. This is no longer the case, and the overhead should now be constant.

New node_modules linkers

As you may know, Yarn is built around a few interfaces. One of them is called a “linker”, and tells Yarn how to install packages on disk. It’s how we can support both PnP and node_modules installs without changing much code.

One advantage of this architecture is how it allows us to efficiently iterate on alternative install strategies. For this release, larixer implemented a new experimental nmMode setting that can be used to instruct the linker to use a specific copy scheme:

• hardlinks-local will use hardlinks when the same package is found multiple times within the same project (but only if they have exactly the same version at the moment).

• hardlinks-global will use hardlinks on identical files (even across different versions!), but will also make them point to a global content-addressable directory. This is similar to what pnpm does. Note that if the cache is corrupted (for example because you manually edited it), Yarn will automatically repair it on subsequent installs.

I myself have been playing with a pnpm-style linker. It hasn’t shipped yet since I’m cautious about adding complexity that could end up unmaintained, but given how small it is there’s a decent chance we could add it in a later release as an experimental install mode.

Improved Shell

As you may know, given that system shells are rarely portable across Windows and Posix, Yarn no longer uses them to run your scripts entries. Instead, we use our very own shell interpreter.

We’re happy to report that this shell just got smarter, and now provides two additional syntaxes that you can reliably use on both Windows and Posix:

build-js & build-css &    # Background jobs
ls 2>/dev/null            # File descriptor redirections

Additionally, background jobs have their output color-coded, so you can clearly identify their output, even interlaced.

ESBuild support

We now use ESBuild to generate the Yarn bundles and as such worked to ensure good compatibility with Plug’n’Play installs. The result is the new @yarnpkg/esbuild-plugin-pnp package which lets you transparently build your code using the default Yarn installation mode. It’s still relatively young, so feel free to drop us an issue if you notice something strange!

While it won’t change much for most end-users, the move to ESBuild also provided decent build speed improvements (around 6x faster), making it less frustrating to build Yarn from sources ✨

New plugin APIs

Yarn supports writing plugins that can inject themselves into various places and leverage some of the builtin modules provided by the core. While we didn’t get the chance to make all the improvements we hoped, we’ve still been able to upgrade the command line framework to Clipanion 3, which lets you write intuitive type-checked commands with a minimal syntactic overhead.

Conclusion

According to DEV.to, those change notes take about 5 minutes to read - by contrast, the 2.0 release post was a whopping 15 minutes! Of course, this time around we didn’t need to fully rewrite Yarn, hence a lower amount of “critical information” we need you to be aware of 😉 We expect that to be the norm from now on: majors won’t have a lot of super impactful changes, mostly just some architecture cleaning and modernization, as new features will tend to land in minors.

As for our team, we’re very happy of the work we’ve been doing! Working on the codebase still feels like a treat, and features are often constrained to a few identifiable files - proving that our initial redesign bet was right. Our stats suggest that the result are visible to our users as well, and while I remain cautious about popularity metrics it’s certainly nice to see.

Finally, remember that Yarn now has an OpenCollective / GitHub Sponsors! If your company benefits from our work, or would like to see particular fixes land, sponsoring the project is a good way to engage with us 🙂

What’s to come?

A few features initially slated for 3.0 have been pushed back to the next minor so that we have more time to properly incubate them. Some of the things we have in motion:

• Corepack integration
• ESM support under PnP mode
• Builtin CLI completion
• Changelog generation
• Improved performances
• pnpm-style linker
• And more…!

Of course that’s only on the top of my head, so it’s possible our objectives shift during the next weeks depending on our own priorities - and of course depending on whether you help us or not 😛

Maël Nison

Lead Yarn maintainer

July 26, 2021

7 min read

Release: Yarn 2.4 🎄🎁

Hey everyone! It’s this time of the year where everyone is slowly preparing for the holidays. This year will probably be slightly different, but I can’t wait to at least take a well deserved time off. But before that, let’s talk about our next minor Yarn release, and a little bit about the next-next release: Yarn 3!

Plugins

We’ll try to reference external plugins made by our community in our release notes, so if you made one that you want to share, please ping us! We’re also looking at adding a page on our website to list them all, improving discoverability 💫

For now, let me present those two:

• yarn.build by ojkelly is a fast monorepo builder for Yarn. In a sense it’s similar to yarn workspaces foreach but more opinionated, and thus easier to adapt to existing workflows. It parallelises builds, shows what’s being executed, and generates zipped archives suitable for AWS and similar platforms.

• prod-install by Larry1123 and NETSVS is a much more powerful version of yarn workspaces focus that copies the selected workspaces into a target location before transforming it to become self-sufficient - the final directory thus being ready to be efficiently cached and deployed via Docker layers.

Audits

Both Yarn 1 and npm had this handy little feature called audit. Originally developed by npm when they acquired Lift, this command lets you quickly check whether some of your dependencies have known vulnerabilities, which may come in handy in some types of application. Unfortunately, since the audit endpoint isn’t documented, its implementation wasn’t entirely obvious.

Thanks to our contributors, Yarn 2.4 now includes proper audit, available via the yarn npm audit command! And to make up for the delay, we’ve implemented various interesting ways to run it, under the form of the -A,--all and -R,--recursive options - check the examples for details!

We’ve also significantly improved the output to be more in line with the rest of the CLI, providing information in a more compact way:

This new output is compatible with the --json flag, meaning that you can leverage the information obtained from yarn npm audit --json from any script you want - even the command-line itself, using tools like jq!

Better Warnings

Peer dependencies have always been a difficult concept to grasp. They are not that hard per se (a peer dependency is always satisfied by the exact package instance used by the parent of the package listing it), but various other factors played into it and caused typical installs to produce many rarely actionable warnings.

No more!, do we say. Starting from 2.4, you can expect the warnings produced by Yarn to become vastly better than what we used to report. For this first release with warnings being a focus, we’ve implemented a new range merging algorithm that lets us drastically decrease the amount of warnings we emit. The idea is simple: imaging the following dependency tree:

.
└── your project/
    ├── @storybook/react/
    │   ├── (peer) react@^15
    │   ├── storybook-plugin-foo/
    │   │   └── (peer) react@^15
    │   └── storybook-plugin-bar/
    │       └── (peer) react@^15
    └── react@17

Before, these are the warnings you’d have had:

your project provides react@17 to @storybook/react, which isn't compatible with react@^15
your project provides react@17 to storybook-plugin-foo, which isn't compatible with react@^15
your project provides react@17 to storybook-plugin-bar, which isn't compatible with react@^15

From all those warnings, only one was truly actionable: the @storybook/react one. The two others were mere byproducts from the first, and were just making the output harder to read. This is now fixed, and Yarn will instead report:

your project provides react@17 (pXYZ), which doesn't satisfy what @storybook-react and its dependents request

The pXYZ is a hash that you can use with a new command, yarn explain peer-requirements <hash>, to get the exact list of packages that contribute to the final peer dependency requirement, and whether they are met or not. For instance, this is what I get in one of my projects:

Log Filters

Even if warnings will get smarter, there’s always this one case where you really don’t care about a specific message. For instance the message saying that a package wasn’t in the cache is sometimes controversial, with half of our users liking it, and the other half wanting to hide it.

While you could use preferAggregateCacheInfo to tweak that, it’s only about one message. What about others? Well, starting from 2.4 we introduce a new setting called logFilters. It has the following syntax:

logFilters:
  - code: YN0005
    level: discard

With this configuration, all messages matching the specified code (which would be builds being disabled, per our documentation) will be removed from the output. And if you need to only remove a single line, it’s possible as well:

logFilters:
  - text: "core-js@npm:2.6.11 lists build scripts, but its build has been explicitly disabled through configuration."
    level: discard

We hope this feature will let you tune your package managers to watch what you truly care about, which can be different from one person to the other.

And also

As always, these release notes focus exclusively on the big-picture stuff - as always, there’s a lot more things that have been improved under the hood. Check our changelog for a comprehensive list, but we can mention:

• Updated our patches to account for TS 4.1 and FSEvents 2.1.2
• Improved usability when using the global cache
• Improved usability in the VSCode ZipFS extension
• Improved performances on recurrent installs
• Improved Windows compatibility when running binaries
• Improved the display for yarn upgrade-interactive
• Fixed the postinstall scripts run by yarn workspaces focus
• Fixed some edge cases with || and interpolation errors
• Added support for proxy settings (caFilePath, …)
• … and more!

What about Yarn 3?

This is a big news for us! Yarn 2.4 is expected to be the last minor release for the 2.x line! After a year of development, we now have put aside enough items to be worth addressing in a new major.

While the 3.x branch will be much less disruptive than the jump from 1.x to 2.x (after all we won’t need to rewrite the whole codebase this time! 😁), it’ll include a few breaking changes. Most of those are listed here but, as you’ll see, they are mostly about old workflows being deprecated, and are unlikely to affect most codebases.

One important note though: given that Node 10 will reach its end-of-life in April, it’s likely that Yarn 3 will be Node 12+ only. So if you want to prepare for it, start considering upgrading to either Node 12 or, better yet, 14!

Maël Nison

Lead Yarn maintainer

November 30, 2020

6 min read

Release: Yarn 2.3 🍦✨

Howdy! Another big month just went by, 2020 confirming being a very weird year for everyone. I hope things will be ok for you, wherever you are.

As for Yarn itself, we’re happy to meet you again to talk a bit about the highlights for the work we’ve done in the third minor of the Yarn 2 release line! Remember that we try to limit these blog posts to about three core items, and that the exhaustive list will always be in our repository. Check it out sometime, there’s some very good stuff there too! 📝

Don’t know how to upgrade? It’s easy: just run yarn set version berry in your project, and you’ll get the latest build. Want to skip the upgrade? Just revert the changes!

Info command

Every now and then, we have this dependency that we want to know more about. We want to know its authors, we want to know its license, we want to know its size … there’s a lot of thing we want to know! And sometimes, we want to retrieve those information from many different packages at once.

Yarn already provides the yarn npm info command, but this command is a bit special in that it prints by default the latest info from the npm registry … and that’s not necessarily what we are using!

To address some of the problem with this command, we’re now introducing a new top-level command, yarn info. It looks like this:

First, wow, that’s pretty! But there are a few interesting things we can note about this display:

• It prints by default the information based on what’s currently used by the active workspace. I could use the -A,--all or -R,--recursive flags to change that, though!

• It prints less information than yarn npm info. For instance, there’s little point in printing the README content as a raw single-line string, so we removed it. By default, yarn info will only print the most relevant information.

• But it prints more information than yarn npm info as well! For instance, because we passed the --cache flag, it also reported the size of the package in the cache, and its exact location.

There are many other gems in the command. By passing the --manifest flag you also get additional fields like the license or the homepage. By passing the --json flag you generate a stream of data that can be easily transformed using jq. You can even add your own data sections if you want, by using our plugin system! Ever wanted a place to show the number of downloads for your dependencies? Their CVEs? Their maintainers? Just use the provided hook, and all those information are yours to give!

Option Documentation

You might not be aware of it, but Yarn uses a pretty unique CLI framework: Clipanion. Very few tools have as much requirements as we do, and it was very important for us to be able to fix bugs and implement features without decreasing our velocity.

In the latest Clipanion update, our contributors implemented a syntax to individually document options. Another one took this new feature, and went over every command, documenting each option one by one. The result looks absolutely great:

Because our CLI is the source of our website’s documentation, you can find the exact same information online. We hope this effort will prove useful to you, as you discover new features you weren’t even aware of until now!

Nohoist

As package manager authors, we try to do our best to support the ecosystem, sometimes going as far as building features just to help one single large project migrating to better practices. In 2017, in order to let React Native users use our newly released workspaces, we implemented a feature called nohoist.

Nohoist was a bit weird. It accepted glob patterns, and presumably the paths matching this glob pattern couldn’t be hoisted. But what if their ancestors were hoisted? Was it meant to support targeting deep packages? After all, it was really only meant to help React Native users in one specific case. Because the feature itself wasn’t entirely clear, it suffered from many bugs over the years, where noone really knew what to do of it. In Yarn 2.0, we decided to completely remove it.

Now, the problem is, React Native still doesn’t support workspaces without help. And we like React Native users. So we’ve been looking for a way to reintroduce something similar to nohoist, but in a way that actually made sense to us. That’s where we introduce you to hoisting limits:

nodeLinker: node-modules
nmHoistingLimits: workspaces

By configuring the nmHoistingLimits setting to workspaces when using the node_modules linker, Yarn will prevent packages from being hoisted past the workspaces that transitively depends on them. In practice, it means that you don’t need to care about the specific hoisting glob patterns: just declare where the hoisting limit is, and Yarn will take care of the rest.

This design is interesting, because it allows us to support one additional feature: “safe hoisting”. See, one problem with the classic hoisting is that it makes it very likely that you’re going to eventually start referring to dependencies without explicitly listing them. Then your users install your packages, and all hell breaks loose.

By configuring nmHoistingLimits on dependencies, Yarn will prevent packages from being hoist past their transitive top-level dependent. It may seem a bit arcane, said like this, but it’s actually quite simple! Imagine the following project:

With the default hoisting, it would turn into the following, mistakenly letting you access all dependencies as your own:

With nmHoistingLimits set on dependencies, Yarn will instead generate the following, ensuring that you won’t ever be able to mistakenly require dependencies you don’t list as your own:

Of course it has its own drawbacks, since the imperfect deduplication also means an heavier disk footprint and slower installs, but it may provide a good safety valve until you can migrate to standard PnP installs.

What’s to come?

With Hacktoberfest coming, now is as good a time as ever to let you know about our issues labelled Good First Issues! In fact, we wrote a whole article about it a few days ago.

As for the features we’re planning for Yarn 2.4, our focus are currently on:

• Adding back yarn audit with revamped output
• New changelog generation capabilities
• PnP support for the exports field, and ESM in general
• And more…!

Of course that’s only on the top of my head, so it’s possible our objectives shift during the next weeks depending on our own priorities - and of course depending on whether you help us or not 😛

One very long-term topic we’re starting to explore are package support for non- JavaScript languages (think C++, Python, Rust, PHP, …). We already have a few ideas (we have an experimental branch generating CMake files, and another contributor played with Python), and we’ll keep evaluating the work needed to get there during the next few months. If you’re familiar with any of those ecosystems and are interested in helping Yarn become the universal package manager, please contact us on Discord!

Maël Nison

Lead Yarn maintainer

September 2, 2020

6 min read

Release: Yarn 2.2 🚅🌟

I hope you enjoyed the summer! As for us, we’ve been hard at work, and this update comes with its good chunk of improvements in various aspects. As usual we keep a detailed list in our repository, but let’s go over the highlights!

Don’t know how to upgrade? It’s easy: just run yarn set version berry in your project, and you’ll get the latest build. Want to skip the upgrade? Just revert the changes!

Dedupe command

One of Yarn’s core values is predictability. We want you to be confident that your project won’t suddenly change in unexpected ways. The lockfile is a large part of this, ensuring that you always get the same dependencies during install, now or in the future.

To explain what the dedupe command is, I first need to explain a bit the lockfile format. In Yarn, we have descriptors (a combination of package name and range), and we associate them with references (versions). A lockfile essentially stores which reference is linked to a specific range.

So what happens when you add new ranges? For example if you already have lodash@^4.0.0 in your lockfile, resolved to 4.0.0, and suddenly add lodash@^4.1.0? Since this new range isn’t compatible with the old one, Yarn will need to resolve it on its own - let’s say to 4.1.0. And now is the interesting part - remember when I said that Yarn tries to be predictable, and thus avoid to update things unless ordered to do so? In this case, it means that lodash@^4.0.0 will not be updated to use 4.1.0, even if they’d be compatible. Instead, it will keep using whatever else it was using before, meaning that you’ll end up with both 4.0.0 and 4.1.0 in your tree.

Functionally this isn’t a problem, because both ranges will use versions compatible with what they advertise. In practice however, it may cause your lockfile to grow needlessly over time as it starts referencing multiple copies of packages, despite the fact that they would have been compatible if the lockfile had been allowed to make wider changes.

The new yarn dedupe command is our solution to that. By default, it will apply a resolution pass that will go over each range and use the highest compatible version that’s already in the lockfile. This has various advantages:

• It doesn’t require the network, so very fast
• In the end, most duplicates will be removed
• It’s very predictable: the highest version wins

Of course, if you have incompatible ranges (for example ^1 and ^2), they won’t be deduped together, since that would lead to invalid trees. In this case, you’ll have to fix your dependencies to remove references to the older range.

Finally, if you want this kind of check to happen on your CI, the -c,--check option will cause the dedupe algorithm to report an error if optimizations would be possible.

Performances

Better performances lead to better UX, and Yarn is a lot about a good UX. To this end, we’ve done various improvements in the 2.2 to improve the performances on real-world projects. For instance, Gatsby on cold cache went 92s → 83s, and 17s → 13s on hot cache.

And because we think we should do better than flaunt about perf increases without live numbers to back them up and publicly track regressions, we’ve setup a live dashboard with our friends at Datadog that shows the results of the daily benchmarks we run against most common package managers. We’re pretty happy about the results!

Note that Yarn currently does a bit more work than its siblings on cold cache installs because we need to convert the registry archives in zip format, more suitable for the usage we have. As registries get better at this, we expect cold cache performances to drastically improve 🚅

Size

Since we’re recommending checking-in the Yarn binary in your repository, we better be careful about how large we are. Our team made various improvements in this regard, and Yarn 2.2 is now exactly 1.8MB large. To give you an idea:

• Yarn Classic is ~5MB large
• pnpm is 35MB
• npm is 61MB

So, yeah. 1.8MB is nice, isn’t it? 🙂

Telemetry

One interesting change in the v2 is that we’re going to enable basic opt-out telemetry. The full details are here, but the gist is that we hope this will allow us to spend more time working on Yarn itself, and with a better understanding of how it’s used in our community at large - which will then help inform the tradeoffs we make.

The telemetry payload is easily opt-out, and we’re committed to send as little information as possible. As soon as the data starts flowing we plan to build public dashboards (similar to our benchmarks) that will help everyone get a better picture of the project.

Other works

Smaller Improvements

This is only a very short list, as always please look at our official changelog for a comprehensive list, but the 2.2 also ships with:

• The shell script language now supports more syntaxes (shell groups { echo foo; echo bar } > bar, basic arithmetic $(($RANDOM + 10)))

• The --immutable flag now accepts an immutablePatterns settings that you can use to define additional paths that aren’t allowed to change during an install - useful to prevent changes to .pnp.js or other artifacts

• Packages referenced via the file: protocol will now update when running yarn add again (they’re still stored in the cache - prefer portal: if you want a symlink-like behavior).

• The new publishConfig.executableFiles field lets you define paths in your package that should be flagged as executable. By default, since Windows has no way to express the executable flag, only files referenced in the bin field will be marked as such, but sometimes you might need others.

• Error messages have been clarified in various contexts, such as when accessing Node builtin within Webpack’s browser context, when running yarn add on unknown packages, or when a lingering package.json exists in a parent directory.

Website

Multiple improvements were made on the website. In particular:

• The migration guide now features a step-by-step section that should help migrate without having to read the entire documentation beforehand.

• The search engine now covers both the manifest and yarnrc pages, making it easier to find information about specific fields.

”Package manager manager”

We are starting discussions with the Node TSC to bundle Yarn with Node in some capacity (the current plan is to ship a shim that would, in turn, install Yarn transparently the first time you call it). The full proposal can be found on the following repository: arcanis/pmm. We strongly advise that you play with it and let us know what you think!

As often, this kind of change benefits from wide support, so if you use Yarn (or pnpm), please feel free to follow the discussion and contribute when relevant. If you don’t use either, remember that others do, and shutting the proposal down purely because you wouldn’t directly benefit from it may not be representative of an inclusive community.

What’s to come?

We’ll try to make more regular minor releases from now on, shipping exactly one minor per month (eventually leading up to the release of Yarn 3 in January 2021). Some topics we have in mind for the next one (come help us! we have a lot of Good First Issues!):

• All-new yarn info
• New changelog generation capabilities
• PnP support for the exports field, and ESM in general
• And more…!

Of course that’s only on the top of my head, so it’s possible our objectives shift during the next weeks depending on our own priorities - and of course depending on whether you help us or not 😛

One very long-term topic we’re starting to explore are package support for non- JavaScript languages (think C++, Python, Rust, PHP, …). We already have a few ideas (we have an experimental branch generating CMake files, and another contributor played with Python), and we’ll keep evaluating the work needed to get there during the next few months. If you’re familiar with any of those ecosystems and are interested in helping Yarn become the universal package manager, please contact us on Discord!

Until then stay safe, wear a mask, and see you next month 😉

Maël Nison

Lead Yarn maintainer

August 28, 2020

7 min read

Release: Yarn 2.1 🐱‍🏍

How are you doing since January? So many things happened since then. I hope you’re all safe, wherever you are.

As for today, we’ll be here to talk about Yarn. And as far as Yarn goes I’m happy to report that our work continued at a very good pace! So good in fact that it’s now time to release the next minor build, the 2.1 🎉

Still, don’t let this little number trick you: more than 350 pull requests were merged since the previous release! This is an incredible pace for our project, only made possible by the dedicated community that gathered around our favorite tool 🌟

So what’s in the 2.1? Many, many things! We’ll go over the main items, but a more detailed list can be found in our repository. You should check it out too, there’s a lot of interesting tidbits!

Don’t know how to upgrade? It’s easy: just run yarn set version berry in your project, and you’ll get the latest build. Want to skip the upgrade? Just revert the changes!

Linker improvements

Node-modules linker

Some people can’t migrate to Plug’n’Play installs just yet. That’s fine! Some of our contributors don’t use it! Yarn supports node_modules installs too! And thanks to Larixer’s impressive work, we’re happy to report that even large and complex repositories have successfully upgraded to Yarn 2. And when I say large, I mean freaking massive ones 🤓

In fact, our position is now that the node_modules linker in Yarn 2 is a strict improvement over the v1. Multiple hoisting issues have been identified and fixed, and the workspace support has also been improved significantly.

To give you an idea, back in the v1, Babel had never been able to use the stock Yarn workspaces. We all wanted it to happen, but because of the very tricky nature of self-hosted compiler repositories, it proved very challenging. Until now! The Babel and Jest repositories are now powered by Yarn 2, and that’s frankly the best seal of quality we could hope for.

So if you’re still on the fence about Plug’n’Play … don’t use it for now! Just migrate for all the other speed and stability and UX improvements 😃

Loose mode

The Plug’n’Play linker also improved, with the introduction of the Loose Mode. In Loose Mode, Yarn will simply warn should the runtime make an unsafe module access, avoiding to throw hard exceptions. This works because we generate at install-time the hoisting map that would have been generated by the node-modules linker, then we use that as a fallback pool for any unspecified dependency. It’s still unsafe, but now you can quickly get a bird’s eye view of all the potential problems without having to fix them all immediately.

Note that the loose mode isn’t enabled by default because, somewhat ironically, it may lead to more verbose executions than the strict mode depending on various factors. In particular, packages that wrapped optional require calls between try/catch blocks won’t be able to prevent the warnings from being emitted, thus causing false positive.

Major improvements to the git: protocol

Workspace cloning

For the past years, most projects have typically followed an “edit, commit, push, release” workflow - the first three parts happening on GitHub while the fourth one was being delegated to the npm registry. Downloading dependencies from Git was always an option, of course, but it didn’t always received the attention it deserved. In particular, cloning specific packages from monorepos was still an unsolved problem.

With Yarn 2.1, this situation changes. Yarn is now able to clone any workspace from any Yarn project. Note that this only works with Yarn projects at the moment due to the lack of yarn workspace <name> run build counterparts on current npm and pnpm releases.

Respectful builds

See, there’s a very important misconception that we (as in, the package manager authors, collectively speaking) have failed to address during the past years. They are not interchangeable. You cannot use X instead of Y and expect a reproducible build. Regardless of what the advertisement says, each package manager has its own feature set, and to expect them all to be in sync is fruitless. We sometimes implement features we like from other package managers, of course, but when all is said, each project still has its own characteristics that others will never truly replicate. And that’s fine!

So what does that mean for Git builds in particular? Imagine, you want to use a project that’s maintained by someone using pnpm. That’s fair. Well, until now, if you were referencing this project with a git: dependency, Yarn would clone it, then run yarn install, then yarn pack. All good! But wait … did it run yarn install? Why not pnpm install? Turns out, there were no good reasons. Package managers aren’t interchangeable, as I was saying. If a project is configured with a pnpm-lock.yaml, then using Yarn to install it is wrong, and would lead to unpredictable builds. Clearly, that’s not an acceptable behaviour.

And so we fixed it! Yarn will now properly detect which packages managers are meant to be used by projects cloned from git dependencies. If there’s a yarn.lock, it’ll be Yarn. If there’s a pnpm-lock.yaml, it’ll be pnpm. And if there’s a package-lock.json, npm it is.

CLI Improvements

Readability

The output was very verbose, sometimes hiding important information (especially on CI, where the cache is either always there or never there). Various changes were made to streamline the output and make it easier to digest.

• On terminals, only five fetch notifications will be displayed at a time. The sixth one will cause the removal of the oldest one, and so forth.

• On CI, Yarn will now print a one-line summary instead of the whole definition (unless configured otherwise).

• A new optional setting, preferTruncatedLines, will ensure that infos and warnings only take a single line each, keeping your output clean and tidy.

• Most CI systems will offer fold groups on each Yarn step. We’re still tweaking a bit this behaviour, and we encourage CI maintainers to reach out to us if you wish to discuss better integrations in this area.

Focused workspaces

The yarn workspaces focus command is a new addition inspired by a 1.x feature of the same name. It allows you to only install the dependencies from one specific workspace (plus its own workspace dependencies), thereby decreasing the install size by a significant factor. Coupled to the --production flag, it’s a great tool for developers looking to integrate monorepos with Docker images.

By the way, the focus implementation takes exactly 99 lines of code. If you’re curious what a plugin looks like, it’s a prime example to keep in mind 😉

Deep accesses from yarn config get/set

The yarn config get/set commands now accept deep paths (ie foo.bar), allowing you to access settings with different levels of granularity.

Additionally, the configuration will now always be redacted before being printed (unless requested otherwise), thereby preventing secrets from accidental leaks.

Meta improvements

Cache filenames

Our cache filenames used to be versioned using a global cache key. As a result, each time we had to bump the cache key (for example because we fixed an issue in the tarball conversion algorithm), all file names changed and were causing a fairly large noise in the Git history for people using zero-installs.

This isn’t the case anymore, as we made the cache content-indexed. Each file will only ever change if the archive content actually changes! 💫

Playground

One of our contributors put CodeSandbox and Yarn together in an impressive playground. Through it, you can easily build reproduction cases for bugs you encounter, decreasing the time needed for us to understand and fix them.

Documentation index

Thanks to Algolia, the Yarn website is now indexed and can be searched from the status bar. We hope this will allow you to quickly find any information you’re looking for - whether it’s authentication configuration, gitignore examples, or lexicon entries.

VSCode Zip Filesystem

We’ve published the Zip FS extension on the VSCode Marketplace. Thanks to the work from Matt Penrice, using the Jump to Definition feature with the extension installed will properly send you to the right files, opened straight from the zip archive.

Note that VSCode has an internal limitation preventing the TypeScript server to cover the files located within zip archives (ie you can Jump to Definition from your sources to zip files, but TypeScript won’t show its types once you get there). Please upvote the following issue to raise the ticket’s priority (we already made a PR, but it unfortunately got rejected).

Other improvements

Performances

• The Plug’n’Play runtime has been further optimized, which may yield significant boost in some cases (in particular ESLint when using the eslint-plugin-import package).

• The binary size also received a lot of attention, and the 2.1 Yarn binary now takes 2.35MB, vs 2.91MB for the 2.0.

### Ecosystem

• Packages can now declare they need to be unpacked in order to be functional using the new "preferUnplugged": true field in the manifest. This will hurt the experience of your users (your project will require hard installs, meaning a heavier footprint and slower installs), so please refrain using this field unless there’s absolutely no other choice.

What’s to come?

We’ll try to make more regular minor releases from now on, shipping exactly one minor per month (eventually leading up to the release of Yarn 3 in January 2021). Some topics we have in mind for the next one (come help us! we have a lot of Good First Issues!):

• Add a new dedupe command to optimize dependency trees
• Add changelog support to the builtin release workflow
• Add support for the exports field
• Add yarn list & yarn fund
• Add the telemetry support (RFC)
• And more…!

Of course that’s only on the top of my head, so it’s possible our objectives shift during the next weeks depending on our own priorities - and of course depending on whether you help us or not 😛

Until then stay safe, wear a mask, and see you next month!

Maël Nison

Lead Yarn maintainer

July 9, 2020

9 min read

Release: Yarn 2.0 🧶🌟

Hi everyone! After exactly 365 days of very intensive development, I’m extremely happy to unveil the first stable release of Yarn 2. In this post I will explain what this release will mean for our community. Buckle up!

If you’re interested to know more about what will happen to Yarn 1, keep reading as we detail our plans later down this post: Future Plans. If you just want to start right now with Yarn 2, check out the Getting Started or Migration guides.

Release Overview

Describing this release is particularly difficult - it contains core, fundamental changes, shipped together with new features born from our own usage.

Highlights

• The output got redesigned for improved readability
• designed for improved readability
• Our CLI commands (yarn add, …) are now aware of workspaces
• Running yarn install can be made optional on per-repo basis
• A safer npx counterpart called yarn dlx to run one-shot tools
• Run commands on all workspaces with yarn workspaces foreach
• Packages can be modified in-place through the patch: protocol
• Local packages can be referenced through the new portal: protocol
• A new workflow has been designed to efficiently release workspaces
• Workspaces can now be declaratively linted and autofixed

But also…

• Package builds are now only triggered when absolutely needed
• Package builds can now be enabled or disabled on a per-package basis
• Scripts now execute within a normalized shell
• Peer dependencies now work even through yarn link
• The lockfile is now proper YAML
• The codebase is now full TypeScript
• Yarn can now be extended through plugins

Breaking changes…

• Configuration settings have been normalized
• Packages must respect their boundaries
• Bundle dependencies aren’t supported anymore
• Packages are stored in read-only archives

Those highlights are only a subset of all the changes and improvements; a more detailed changelog can be found here, and the upgrade instructions are available here.

Frequently Asked Questions

Who should we thank for this release?

A significant amount of work has been done by larixer from SysGears, who crawled deep into the engine with the mission to make the transition to Yarn 2 as easy as possible. In particular he wrote the whole node_modules compatibility layer, which I can tell you is no easy feat!

My thanks also go to everyone who spontaneously joined us for a week or a month during the development. In particular embraser01 for the initial Windows support, bgotink for typing our filesystem API, deini for his contributions to the CLI, and Daniel for his help on the infrastructure migration.

This work couldn’t have been possible without the support from many people from the open-source community - I think in particular to Nicolò from Babel and Jordan from Browserify, but they’re far from being the only ones: the teams of Gatsby, Next, Vue, Webpack, Parcel, Husky, … your support truly made all the difference in the world.

And finally, the project lead and design architect for Yarn 2 has been yours truly, Maël Nison. My time was sponsored in large part by Datadog, which is a super dope place to develop JS (which is hiring 😜), and by my fiancé and our cats. Never forget that behind all open-source projects are maintainers and their families.

How easy will it be to migrate to Yarn 2?

Thanks to our beta testers and the general support of the ecosystem we’ve been able to soften a lot the pain associated with such a major upgrade. A Migration Guide is available that goes into more detail, but generally speaking as long as you use the latest versions of your tools (ESLint, Babel, TypeScript, Gatsby, etc), things should be fine.

One particular caveat however: Flow and React-Native cannot be used at the moment under Plug’n’Play (PnP) environments. We’re looking forward to working with their respective teams to figure out how to make our technologies compatible. In the meantime you can choose to remain on Yarn 1 for as long as you need, or to use the node_modules plugin, which aims to provide a graceful degradation path for smoother upgrade (note that it’s still a work in progress - expect dragons). More details here.

If you don’t want to upgrade all of your projects, just run yarn policies set-version ^1 in the repositories that need to stay on Yarn 1, and commit the result. Yarn will always prefer the checked-in binaries over the global ones, making it the best way to ensure that everyone in your team shares the exact same release!

What will happen to the legacy codebase?

Yarn 1.22 will be released next week. Once done, the 1.x branch will officially enter maintenance mode - meaning that it won’t receive further releases from me except when absolutely required to patch vulnerabilities. New features will be developed exclusively against Yarn 2. In practical terms:

• The classic repository (yarnpkg/yarn) will move over to yarnpkg/classic to reflect its maintenance status. It will be kept open for the time being, but we’ll likely archive it in a year or two.

• The modern repository will not be renamed into yarnpkg/yarn, as that would break a significant amount of backlink history. It will remain yarnpkg/berry for the foreseeable future.

• The old website will move over to classic.yarnpkg.com, and the new website (currently next.yarnpkg.com) will be migrated to the main domain name.

• The yarn package on npm will not change; we will distribute further version using the new yarn set version command.

We expect most of those changes to be completed by February 1, 2020.

In Depth

CLI Output

Back when Yarn was released its CLI output was a good step forward compared to other solutions (plus it had emojis! 🧶), but some issues remained. In particular lots of messages were rather cryptic, and the colours were fighting against the content rather than working with it. Strong from this experience, we decided to try something different for Yarn 2:

Almost all messages now have their own error codes that can be searched within our documentation. Here you’ll find comprehensive explanations of the in-and-outs of each message - including suggested fixes. The colours are now used to support the important parts of each message, usually the package names and versions, rather than on a per-line basis.

We expect some adjustments to be made during the following months (in particular with regard to colour blindness accessibility), but over time I think you’ll come to love this new display!

Workspace-aware CLI

Working with workspaces can sometimes be overwhelming. You need to keep the state of your whole project in mind when adding a new dependency to one of your workspaces. “Which version should I use? What’s already used by my other workspaces?”, etc.

Yarn now facilitates the maintenance of such setups through various means:

• yarn up <name> will upgrade a package in all workspaces at once
• yarn add -i <name> will offer to reuse the same version as the ones used by your other workspaces (and some other choices)
• The version plugin will give you a way to check that all the relevant workspaces are bumped when one of them is released again.

Those changes highlight the new experience that we want to bring to Yarn: the tool becomes an ally rather than a burden.

Zero-Installs

While not a feature in itself, the term “Zero Install” encompasses a lot of Yarn features tailored around one specific goal - to make your projects as stable and fast as possible by removing the main source of entropy from the equation: Yarn itself.

To make it short, because Yarn now reads the vendor files directly from the cache, if the cache becomes part of your repository then you never need to run yarn install again. It has a repository size impact, of course, but on par with the offline mirror feature from Yarn 1 - very reasonable.

For more details (such as “why is it different from checking in the node_modules directory”), refer to this documentation page.

New Command: yarn dlx

Yarn 2 introduces a new command called yarn dlx (dlx stands for download and execute) which basically does the same thing as npx in a slightly less dangerous way. Since npx is meant to be used for both local and remote scripts, there is a decent risk that a typo could open the door to an attacker:

$ npx serv # Oops, should have been "serve"

This isn’t a problem with dlx, which exclusively downloads and executes remote scripts - never local ones. Local scripts are always runnable through yarn run or directly by their name:

$ yarn dlx terser my-file.js
$ yarn run serve
$ yarn serve

New Command: yarn workspaces foreach

Running a command over multiple repositories is a relatively common use case, and until now you needed an external tool in order to do it. This isn’t the case anymore as the workspace-tools plugin extends Yarn, allowing you to do just that:

$ yarn workspaces foreach run build

The command also supports options to control the execution which allow you to tell Yarn to follow dependencies, to execute the commands in parallel, to skip workspaces, and more. Check out the full list of options here.

New Protocol: patch:

Yarn 2 features a new protocol called patch:. This protocol can be used whenever you need to apply changes to a specific package in your dependency tree. Its format is similar to the following:

{
  "dependencies": {
    "left-pad": "patch:left-pad@1.3.0#./my-patch.patch"
  }
}

Together with the resolutions field, you can even patch a package located deep within your dependency tree. And since the patch: protocol is just another data source, it benefits from the same mechanisms as all other protocols - including caching and checksums!

New Protocol: portal:

Yarn 2 features a new protocol called portal:. You can see portal: as a package counterpart of the existing link: protocol. Where the link: protocol is used to tell Yarn to create a symlink to any folder on your local disk, the portal: protocol is used to create a symlink to any package folder.

{
  "dependencies": {
    "@my/app": "link:./src",
    "eslint-plugin-foo": "portal:./pkgs/eslint-plugin-foo"
  }
}

So what’s the difference you say? Simple: portals follow transitive dependencies, whereas links don’t. Even better, portals properly follow peer dependencies, regardless of the location of the symlinked package.

Workspace Releases

Working with workspaces brings its own bag of problems, and scalable releases may be one of the largest one. Most of large open-source projects around here use Lerna or a similar tool in order to automatically keep track of changes applied to the workspaces.

When we started releasing the beta builds for Yarn 2, we quickly noticed we would be hitting the same walls. We looked around, but existing solutions seemed to have significant requirements - for example, using Lerna you would have to either release all your packages every time, or to keep track yourself of which packages need to be released. Some of that work can be automated, but it becomes even more complex when you consider that a workspace being released may require unrelated packages to be released again too (for example because they use it in their prepack steps)!

To solve this problem, we’ve designed a whole new workflow available through a plugin called version. This workflow, documented here, allows you to delegate part of the release responsibility to your contributors. And to make things even better, it also ships with a visual interface that makes managing releases a walk in the park!

This workflow is sill experimental, but it works well enough for us that we think it’ll quickly prove an indispensable part of your toolkit when building large projects using workspaces.

Workspace Constraints

Workspaces quickly proved themselves being one of our most valuable features. Countless projects and applications switched to them during the years. Still, they are not flawless. In particular, it takes a lot of care to keep the workspace dependencies synchronized.

Yarn 2 ships with a new concept called Constraints. Constraints offer a way to specify generic rules (using Prolog, a declarative programming language) that must be met in all of your workspaces for the validation to pass. For example, the following will prevent your workspaces from ever depending on underscore - and will be autofixable!

gen_enforced_dependency(WorkspaceCwd, 'underscore', null, DependencyType) :-
  workspace_has_dependency(WorkspaceCwd, 'underscore', _, DependencyType).

This other constraint will require that all your workspaces properly describe the repository field in their manifests:

gen_enforced_field(WorkspaceCwd, 'repository.type', 'git') :-
  workspace(WorkspacedCwd).

gen_enforced_field(WorkspaceCwd, 'repository.url', 'ssh://git@github.com/yarnpkg/berry.git') :-
  workspace(WorkspacedCwd).

Constraints are definitely one of our most advanced and powerful features, so don’t fret yourself if you need time to wrap your head around it. We’ll follow up with blog posts to explore them into details - watch this space!

Build Dependency Tracking

A recurrent problem in Yarn 1, native packages used to be rebuilt much more than they should have. For example, running yarn remove used to completely rebuild all packages in your dependency tree.

Starting from Yarn 2 we now keep track of the individual dependency trees for each package that lists postinstall scripts, and only run them when those dependency trees changed in some way:

➤ YN0000: ┌ Link step
➤ YN0007: │ sharp@npm:0.23.0 must be rebuilt because its dependency tree changed
➤ YN0000: └ Completed in 16.92s
➤ YN0000: Done with warnings in 21.07s

Per-Package Build Configuration

Yarn 2 now allows you to specify whether a build script should run or not on a per-package basis. At the moment the default is to run everything, so by default you can choose to disable the build for a specific package:

{
  "dependenciesMeta": {
    "core-js": {
      "built": false
    }
  }
}

If you instead prefer to disable everything by default, just toggle off enableScripts in your settings then explicitly enable the built flag in dependenciesMeta.

Normalized Shell

Back when Yarn 2 was still young, the very first external PR we received was about Windows support. As it turns out Windows users are fairly numerous, and compatibility is important to them. In particular they often face problems with the scripts field which is typically only tested on Bash.

Yarn 2 ships with a rudimentary shell interpreter that knows just enough to give you 90% of the language structures typically used in the scripts field. Thanks to this interpreter, your scripts will run just the same regardless of whether they’re executed on OSX or Windows:

{
  "scripts": {
    "redirect": "node ./something.js > hello.md",
    "no-cross-env": "NODE_ENV=prod webpack"
  }
}

Even better, this shell allows us to build tighter integrations, such as exposing the command line arguments to the user scripts:

{
  "scripts": {
    "lint-and-build": "yarn lint \"$@\" && yarn build \"$@\""
  }
}

Improved Peer Dependency Links

Because Node calls realpath on all required paths (unless —preserve-symlinks is on, which is rarely the case), peer dependencies couldn’t work through yarn link as they were loaded from the perspective of the true location of the linked package on the disk rather than from its dependent.

Thanks to Plug’n’Play which can force Node to instantiate packages as many times as needed to satisfy all of their dependency sets, Yarn is now able to properly support this case.

New Lockfile Format

Back when Yarn was created, it was decided that the lockfile would use a format very similar to YAML but with a few key differences (for example without colons between keys and their values). It proved fairly annoying for third-party tools authors, as the parser was custom-made and the grammar was anything but standard.

Starting from Yarn 2, the format for both lockfile and configuration files changed to pure YAML:

"@yarnpkg/parsers@workspace:^2.0.0-rc.6, @yarnpkg/parsers@workspace:packages/yarnpkg-parsers":
  version: 0.0.0-use.local
  resolution: "@yarnpkg/parsers@workspace:packages/yarnpkg-parsers"
  dependencies:
    js-yaml: ^3.10.0
    pegjs: ^0.10.0
  languageName: unknown
  linkType: soft

TypeScript Codebase

While it might not directly impact you as a user, we’ve fully migrated from Flow to TypeScript. One huge advantage is that our tooling and contribution workflow is now easier than ever. And since we now allow building Yarn plugins, you’ll be able to directly consume our types to make sure your plugins are safe between updates.

export interface Package extends Locator {
  version: string | null;
  languageName: string;
  linkType: LinkType;
  dependencies: Map<IdentHash, Descriptor>;
  peerDependencies: Map<IdentHash, Descriptor>;
  dependenciesMeta: Map<string, Map<string | null, DependencyMeta>>;
  peerDependenciesMeta: Map<string, PeerDependencyMeta>;
}

Modular Architecture

I recently wrote a whole blog post on the subject so I won’t delve too much into it, but Yarn now follows a very modular architecture.

In particular, this means two interesting things:

• You can write plugins that Yarn will load at runtime, and that will be able to access the true dependency tree as Yarn sees it; this allows you to easily build tools such as Lerna, Femto, Patch-Package, …

• You can have a dependency on the Yarn core itself and instantiate the classes yourself (note that this part is still a bit experimental as we figure out the best way to include the builtin plugins when operating under this mode).

To give you an idea, we’ve built a typescript plugin which will automatically add the relevant @types/ packages each time you run yarn add. Plugins are easy to write - we even have a tutorial -, so give it a shot sometime!

Normalized Configuration

One very common piece of feedback we got regarding Yarn 1 was about our configuration pipeline. When Yarn was released we tried to be as compatible with npm as possible, which prompted us to for example try to read the npm configuration files etc. This made it fairly difficult for our users to understand where settings should be configured.

initScope: yarnpkg
npmPublishAccess: public
yarnPath: scripts/run-yarn.js

In Yarn 2, the whole configuration has been revamped and everything is now kept within a single source of truth named .yarnrc.yml. The settings names have changed too in order to become uniform (no more experimental-pack-script-packages-in-mirror vs workspaces-experimental), so be sure to take a look at our shiny new documentation.

Strict Package Boundaries

Packages aren’t allowed to require other packages unless they actually list them in their dependencies. This is in line with the changes we made back when we introduced Plug’n’Play more than a year ago, and we’re happy to say that the work we’ve been doing with the top maintainers of the ecosystem have been fruitful. Nowadays, very few packages still have compatibility issues with this rule.

// Error: Something that got detected as your top-level application
// (because it doesn't seem to belong to any package) tried to access
// a package that is not declared in your dependencies
//
// Required package: not-a-dependency (via "not-a-dependency")
// Required by: /Users/mael/my-app/
require(`not-a-dependency`);

Deprecating Bundle Dependencies

Bundle dependencies are an artefact of another time, and all support for them has been dropped. The installs will gracefully degrade and download the packages as originally listed in the dependencies field.

{
  "bundleDependencies": ["not-supported-anymore"]
}

Should you use bundle dependencies, please check the Migration Guide for suggested alternatives.

Read-Only Packages

Packages are now kept within their cache archives. For safety and to prevent cache corruptions, those archives are mounted as read-only drives and cannot be modified under normal circumstances:

const { writeFileSync } = require(`fs`);
const lodash = require.resolve(`lodash`);

// Error: EROFS: read-only filesystem, open '/node_modules/lodash/lodash.js'
writeFileSync(lodash, `module.exports = 42;`);

If a package needs to modify its own source code, it will need to be unplugged - either explicitly in the dependenciesMeta field, or implicitly by listing a postinstall script.

Conclusion

Wow. That’s a lot of material, isn’t it? I hope you enjoy this update, it’s the culmination of literally years of preparation and obstinacy.

Everything I believe package management should be, you’ll find it here. The result is for sure more opinionated than it used to be, but I believe this is the way going forward - a careful planning of the long term user experience we want to provide, rather than a toolbox without directions.

As for me, working on Yarn has been an incredible experience. I’m simultaneously project manager, staff engineer, lead designer, developer relations, and user support. There are ups and downs, but every time I hear someone sharing their Yarn success story my heart is internally cheering a little bit. So do this: tell me what you like, and help fix what you don’t.

Happy 2020! 🎄

Maël Nison

Lead Yarn maintainer

January 24, 2020

17 min read