Building a Flutter SDK: A Deep Dive Into pub.dev — Part Two

Introduction

Building a Flutter SDK is a series about crafting your own Flutter packages and SDKs from scratch. The articles in the series put into words the lessons the Stream Flutter team has learned over the years building our own Flutter SDKs. The series uses two of our SDKs as examples which you can take a look at for more context:

• Stream Chat Flutter
• Stream Video Flutter

Part 1: Breaking Down a Flutter Package of the series delved into every aspect of a Flutter package; from exploring the purpose of existing files to publishing your own package. Now that building and publishing a package is understood, we can turn our sights to methods of distribution.

When developers start learning about building apps in Flutter, adding dependencies is one of the first things they cover. While very few projects today can exist without any dependencies, external packages have extra importance in Flutter since several important packages contain code native to each platform.

Until Google decides to push out Fuchsia, all native code will be written in languages other than Dart. Since Flutter supports many platforms, it is unreasonable for any developer to write native code for all platforms. This difficulty is compounded by the fact that even if a developer understands all native languages, every platform handles applications in a different way. For example, Windows can have multiple windows for an application, while Android and iOS can’t. External packages are an important, if not an essential component of all apps. Having a proper distribution source for them is critical to the well-being of the Flutter framework itself.

pub.dev, or often simply “Pub”, is the main Google-hosted repository of all public Flutter packages. If you are building a Flutter SDK, all your packages will likely live on pub, most times exclusively. While there are other ways to publish a package, as seen later and in the previous article, most SDK/package developers prefer simply publishing packages on pub.

In this article, we dive into everything about pub.dev. Let’s start.

About Package Stores

Managing and distributing packages isn’t always an easy task. It might be relatively easy to forget as a Flutter developer, but looking at Android and iOS makes the difficulty a bit more obvious. Most packages on both platforms can be distributed through multiple package distribution sites and also integrated into the project in different ways.

Android libraries are usually published on platforms like Maven Central, the Google Maven Repository, and in the past, JCenter. Developers often use tools like Gradle or Maven to manage dependencies and include these libraries in their Android projects.

For iOS, libraries are commonly distributed through CocoaPods or Carthage. CocoaPods is a dependency manager for Swift and Objective-C Cocoa projects, while Carthage is another dependency manager specifically for Swift projects. Developers can specify dependencies in their project's Podfile or Cartfile, respectively, and then use CocoaPods or Carthage to fetch and integrate these libraries into their iOS projects. Developers can also distribute iOS libraries as binary frameworks or through manual integration.

It is helpful to remember that both platforms have had language transitions in the past; Android went from Java to Kotlin, while iOS went from Objective-C to Swift. While there is proper support for inter-op between the two languages, it may sometimes cause issues when using older packages with newer apps.

Compared to the aforementioned platforms, Flutter (at least for now) has a simpler distribution and integration system for packages. At the time of writing, pub is the only major Flutter package distribution site, even though it is not mandated to fetch packages from pub.

What is pub.dev?

Pub.dev is the official package repository for Dart and Flutter, developed and maintained by the Flutter team at Google. It serves as a central hub where developers can discover, share, and use packages in their Dart and Flutter projects.

Package scores on pub

Since it is fairly difficult to explore the source code of a package every time, there needs to be a way for a developer to judge the quality of a package without spending too much time. There are various ways to do this in general, but nearly all methods have their skeptics. Having a review system that involves a team is often controversial, as any guidelines can be seen as too harsh - especially in an open-source system.

The way pub has settled is a system with three main metrics: two are a set of package metrics that are auto-generated and updated by pub, and the third is a “like” count for each package for the community to show their approval of the package.

In a package, you will see the metrics displayed like this:

While the like count is easy to understand, other metrics on the board are not so obvious. Quoting the Flutter docs, the popularity metric is defined like this:

Popularity measures the number of apps that depend on a package over the past 60 days. We show this as a percentile from 100% (among the top 1% most used packages) to 0% (the least used package). Although this score is based on actual download counts, it compensates for automated tools such as continuous builds that fetch the package on each change request.

The docs also mention that absolute usage counts may be available in the future. Not having an absolute count can often be problematic because package metrics are rather important when building an SDK for a service such as Stream. Having usage stats would give package developers more insight into how many developers download the SDK vs. how many end up fully using it in their app, showing us the ease of use of the SDK and the adoption rate.

Last, the “pub points” metric is an automated score generated from the package. Here is the list of conditions that increase a package’s pub points:

The Flutter team at Google has set a few qualities that are good for an SDK to have:

Following Dart conventions: Having a valid pubspec.yaml, README.md, CHANGELOG.md, and an open-source license. Having a custom license will knock down your package score by 10 points in the current system.

Provide Documentation: You must provide documentation comments (”///” are doc comments) for 20% of your public API in the package. You must also provide an example via an example folder in the package root. You can theoretically just do a single file (example/lib/main.dart) and pub may accept it as a valid example. Still, we would recommend adding a full example for users to test things out and reduce your future issues.

Platform Support: Supporting more platforms with your package gives you extra pub points. Most Dart packages can do this relatively easily, but any plugins will likely require native code on all platforms.

Pass Static Analysis: When your package is uploaded, the pana command is run to check for any static errors in the package. Any errors, warnings, lint errors, or even formatting issues will lower your pub points.

Support Up-To-Date Dependencies: This check validates that all your dependencies and overall package support the latest Dart and Flutter SDKs. To check this, you can run the following command: dart pub outdated --no-dev-dependencies --up-to-date --no-dependency-overrides.

Managing Metadata

Every package contains metadata for any potential users to learn more about the package or access resources such as documentation, websites, issue trackers, and more. Please refer to the [pubspec.yaml](https://getstream.io/blog/breaking-down-flutter-package/) section in the previous article in our series for more about adding metadata to your package. This section covers the visible metadata on the pub site for any package and not all kinds of metadata that can be added to a package.

The main thing you can add in terms of metadata on a package listing are topics that the package covers. For our Video SDK, we wanted to indicate that the package can do video, audio, live streaming, etc., which shows up like this:

To add topics to your Flutter package, add the topics tag at the root of your pubspec.yaml file like this:

topics:
  - video
  - audio
  - audioroom
  - webrtc
  - livestream

Inside the package, your metadata can be seen on the side. Let’s look at the metadata of flutter_bloc as an example:

The metadata contains your package description, homepage, repository, issue tracker, contributing page, topics, and docs. All of these can be set in your pubspec.yaml file, as seen in the previous article in the series.

In addition to this, you can also add screenshots to your package as metadata. These differ from screenshots you may have added to your README.md file. These will be displayed above the metadata like this:

Once opened, you can swipe through all the screenshots and also add descriptions to them:

To add screenshots, you need to add the same field to the root of your pubspec.yaml file:

screenshots:
  - description: The flutter bloc package logo.
    path: screenshots/logo.png

The description of the images can be added as a field to the same tag.

About Flutter Favorites

When reading around the pub website, you may see a program named Flutter Favorites with certain packages listed below like this:

The Flutter Favorites program highlights packages and plugins that are widely used and which the team recommends considering when developing your app. It's important to note that the title does not imply suitability for your own implementation. Always check if the packages and plugins match your own app needs.

Defining a system for recommending a list of packages is often difficult. It is quite easy to think that popular and maintained packages should make the cut; however, several popular packages promote shortcuts and anti-patterns when creating an app that can later stifle development velocity.

Personally, I do not know if there’s a perfect solution to recommending the right packages. But for now, a team of GDEs and prominent Flutter experts selects the packages in the list. Knowing quite a few of them, I’d take their recommendations. 🙂

Here are the qualities that pub lists for having a package added to the program:

• Overall package score
• Permissive license, including (but not limited to) Apache, Artistic, BSD, CC BY, MIT, MS-PL, and W3C
• GitHub version tag matches the current version from pub.dev, so you can see exactly what source is in the package
• Feature completeness—and not marked as incomplete (for example, with labels like “beta” or “under construction”)
• Verified publisher
• General usability when it comes to the overview, docs, sample/example code, and API quality
• Good runtime behavior in terms of CPU and memory usage
• High-quality dependencies

Publishers

Originally, pub had a simple system that associated all packages with an uploader. You could then look at all the packages that the uploader has published. However, every uploader was associated with a single email address. This system did not work well for an organization similar to Stream or Invertase, which has many packages and several people maintaining them. Publishers were the solution the Flutter team devised to solve this problem.

A publisher is an entity tied to a domain that can publish multiple packages and have multiple emails associated with it. It also adds authenticity to your packages and avoids displaying your personal email address. To create a verified publisher, pub.dev uses DNS (domain name system) domains as an identification token.

For example, Stream publishes under the getstream.io publisher, which contains all our packages beneath it.

Unlisting, Discontinuing, Retracting, & Deleting a Package

Situations might require you to stop supporting or completely pull one of your deployed packages from pub. In these cases, there are several things you can do that allow you to discontinue your package gracefully.

Discontinuing a Package on pub

Let developers know you no longer support this package but will keep it up on pub.dev, you can choose to mark it as discontinued. This can be done through the package settings:

A discontinued package has a clear warning for the user that the package is no longer supported and will not receive any updates. Here is an example of the older version of the ‘sensors’ package:

It is always good to give reasons why this package was deprecated and if there are any alternatives for existing users to move to. In the same package, the developers added a deprecation notice for the package:

Unlisting a Package on pub

If you do not want to take the rather drastic action of pulling the package, you can unlist a package instead. An unlisted package does not appear in the general pub listing. You can mark this through the package settings:

Somewhat paradoxically, the pub search allows you to list unlisted packages if you desire. So keep in mind this is a temporary measure to stop developers from discovering the package or, in general, keeping it out of the public view. An unlisted package can still be viewed via the direct link and pulled like a normal dependency.

Retracting a Package on pub

If you realize a mistake on a package version you published, it can be retracted from pub up to seven days after publication. In this case, a version is marked as retracted in pub but can be fetched as a dependency if added in dependency_overrides. This is only done in extreme cases, such as if there is something significant missing in the package. In most cases, a simple patched version can do the trick.

Retraction can be found in the same package settings as the settings in the previous sections:

Deleting a Package

Here is where it gets a bit more complicated. As the message says, when you publish a package, publishing is forever. Pub has no direct mechanism to simply delete a package from pub. This is likely there for a good reason. In all modern projects, all dependencies often depend upon other dependencies, which again do the same. If you take one small brick out from the lowest of these layers, it really can wreak havoc on the entire ecosystem.

The scenario mentioned above might seem like a bit of hyperbole, but I recommend watching this video about when this happened with a small package called 'leftpad'. So, can we delete a package on pub? No… but also, maybe.

There is no direct way to delete a package at all. However, in rare cases, they may do it for you if you send a request to Google directly. While I have not done anything of the sort as of yet, I assume they review it on a case-by-case basis and judge if something warrants deletion. You can see an example of what happened on this GitHub issue.

Pub CLI

The Pub CLI is what we used in the last article to publish a package using the flutter pub publish command. However, there is some depth to what you can do with the pub CLI, and this section explores some options. I was considering if this section should be in the last article, where we discussed a full Flutter package, but I decided on adding it here since it explores pub as a concept. The Pub CLI originally used to be a standalone command, so to publish a package, you would use pub publish instead of flutter pub publish / dart pub publish, but now it is accessed via the flutter or dart commands.

Adding and Removing Packages

To add a package to your pubspec.yaml file without manually adding it yourself, you can use the pub add subcommand:

flutter pub add stream_chat_flutter

Similarly, to remove it, use the pub remove subcommand:

flutter pub remove stream_chat_flutter

Upgrade / Downgrade Dependencies

Over time, dependencies in any project become outdated since newer updates are launched. To upgrade all packages in the project, you can use:

flutter pub upgrade 

Similarly, to find out-of-date dependencies, you can use the pub outdated subcommand:

flutter pub outdated

This will give you all outdated packages and their latest versions:

There is also a downgrade command that downgrades all packages unless a single package is specified:

flutter pub downgrade

Running Global Scripts

You can activate some Dart packages globally rather than for just one single project. The global option in the pub CLI allows you to run Dart scripts from the command line when you are not currently inside a package.

You usually use this for activating a package that you may need globally such as melos :

flutter pub global activate melos

Pub Cache

Pub stores dependencies locally once downloaded in the pub cache. You can also manually add and remove packages from this local cache using the pub cache add and pub cache clean command:

flutter pub cache add stream_chat_flutter
flutter pub cache clean

Sometimes, there can be issues due to these local files changing for one reason or another. In this situation, you can repair the cache to sort this out:

flutter pub cache repair

Pub Token

While pub is currently the only popular way to manage dependencies, you can also use a third-party site to do the same. To do this, you must add the token to authenticate with the third-party site using the pub token subcommand. This section uses examples from the Flutter documentation.

You can add a token using pub token add :

dart pub token add https://some-package-repo.com/my-org/my-repo

Result: Enter secret token: <Type token on stdin>
     Requests to "https://some-package-repo.com/my-org/my-repo" will now be 
     authenticated using the secret token.

You can also list the tokens already added and remove any of them:

flutter pub token list 

flutter pub token remove -all

A Few Issues With

In general, I consider pub a clean, simple, and easy-to-use solution to the problem of handling dependencies in Flutter. However, that does not mean it is a perfect platform. We have faced some problems with Stream, and this section goes into some of them. There are more issues that other people may have encountered, and this is not an exhaustive list.

One major issue we have with pub at Stream has to do with metrics. The popularity metric of pub is a decent measure if you simply want to compare two packages or want to know roughly how popular a package is. However, for companies that make SDKs, it can often be lacking as a true metric of popularity. Something like a direct number of downloads, even if it is just shown to the developers of the package rather than the general public, would be an amazing way to know more about the true popularity of a package. More metrics, such as change over time, would be even better, but let’s start with the basic expectations for now.

Another issue we’ve had, and have also seen several package maintainers having, is the static analysis check on the package. A Flutter package usually supports a range of Flutter SDK versions, and a new Flutter SDK version can introduce breaking changes that can break the package. This will be shown as an analysis issue to all package users, reducing trust.

To compound this problem, we have had several cases where we would have to update the major version and stop support for the existing Flutter package version. There could be any number of potential solutions to this issue, but the lowest effort would likely be to introduce analysis checks for the last few Flutter versions. This would, at the very least, let developers know what versions this package is compatible with, and new Flutter versions would not show the package as broken overnight.

There are also some other things I may disagree with on the package scoring front. Still, I do agree that there is no silver bullet solution to ranking packages, and so the Flutter team will likely be implementing the least objectionable solution to all stakeholders.

Conclusion

While knowing how to develop a package is important, it is also as important to fully understand how to distribute it, maintain it, and, if needed, remove it. We hope this article clarified everything around the pub.dev ecosystem and enabled you to handle package releases better.

Now that we’re done with a breakdown of a package and its distribution, further articles will dive into the best practices of building a package, managing state, architecture, documentation, and more. See you in the next one.