KENT: And we're live with JavaScript Air. Hello, everyone. My name is Kent C. Dodds and I am your host for this JavaScript podcast broadcast, coming at you live from Provo Canyon in Utah. I'm on vacation with my family and my wife was gracious enough to give me this hour of time away to bring to you all some awesome content, from some awesome people, about managing dependencies like a boss. So that is our show today. We have some subject matter experts on that, but before we get into that I'd like to give a shout out to the sponsors that make this show possible.

Our first sponsor is Egghead.io. They have a huge library of bite-size web development training videos. Check them out for content on JavaScript, Angular, React, Node and much more. Egghead.io is also the host of two free courses about Redux from Dan Abramov. Find them at egghead.io/redux.

And Front End Masters is a recorded expert-led workshop with courses on Advanced JavaScript, Asynchronous, and Functional JS as well as lots of other great courses on front end topics. Brian Holt just released a Redux React router, course it's awesome, I hear, so check that out.

And then TrackJS report the bugs in your JavaScript before customers notice them and with the telemetry timeline will have the context to actually fix them. Check them out and start tracking JavaScript errors today at TrackJS.com.

And WebStorm is a powerful JavaScript IDE. Give it a try for a more productive development with ES6, Angular, and React. Use the discount code "JavaScript Air" at checkout at jetbrains.com/webstorm to get 20% off your WebStorm personal subscription.

And Training Technologies is looking for passionate and inventive full stack JavaScript developers who want to work on cutting edge solutions in a collaborative and challenging environment, while helping build the top choice platform for derivative traders. All right, sweet. So that is our sponsors, super grateful for them.

Let's go ahead and, I actually have a couple more announcements. So one other thing that's cool about this show is that because it's live, we can interact with you, the people who are watching live. And so if you have questions about managing dependencies or what our favorite ice cream is, actually, I probably won't answer that question, but maybe. You can ask those kinds of questions with the #JSAirQuestion on Twitter and we'll look at those at the end of the show and run through those.

One other announcement. It's not a normal announcement. It's kind of sad news. Hang Outs on Air, the software that we use to create this show and that does so much for us from Google, is actually being shut down and swallowed into YouTube Live. As far as I know, YouTube Live isn't capable of the video chat features that Google Hangouts on Air are capable of. Correct me if I'm wrong. Because of that I'm looking for an alternative that can do basically everything that Hangouts On Air does for me, so have a chat that's recorded and available live, and then made available on YouTube automatically after. If I can't find something like that then I'll probably shut down JavaScript Air, to be totally honest. And that wouldn't be a terrible thing for me because I'd get some of my life back, but it would be really sad because there's lots of great stuff coming out of JavaScript Air. So if you're interested in JavaScript Air continuing, then please feel free to suggest an alternative to Hangouts On Air for me to look into. But, yeah, technically, they are shutting down. They have a hard shutdown on September 12th, so that might be when the show's over. We'll see. Great, so that's kind of, wipe a tear there.

But so a couple of other announcements, this is a weekly show and next week is a special show. It's gonna be on Friday, not the normal Wednesday, because we're gonna be on sight at React Rally here in Utah. It's gonna be fantastic. React Rally was wonderful last year. It's gonna be even better this year. So we're gonna be talking to attendees and speakers, maybe organizers on sight at React Rally. It's gonna be sweet. So Friday, the 26th of August at 12 PM, Central Time, I think. That time might change, so keep an eye on the website. As always, follows us on Twitter, Facebook, and Google+ to keep up with the latest. And that's that for announcements. So let's go ahead and introduce our guests. First we have Benjamin Coe.

BEN: Hey, I'm, as Kent says, I'm Benjamin Coe and I'm the third employee at npm, where we do a lot of stuff off dependencies and I care a lot about dependencies. In my copious amounts of spare time, I am also one of the lead maintainers of Yargs, everyone's favorite pirate-themed command line parser and Istanbul the code coverage tool. So I have a lot of interest in dependencies and open source software development.

KENT: Awesome, yeah. Historically, I've used commander for my command line tooling stuff. But I've seen other C lines that are using Yargs and I feel like it is a bit more powerful. So I'm gonna look into it on my next CLI project. So cool. And Stephan Bonneman.

STEPHAN: Hi. Yup, I'm joining you from Berlin today. I'm working on Greenkeeper which is a service specifically to manage dependencies. So it's pretty much fitting the topic today. I'm also working on semantic lift, something we talked on an episode one month ago I think. And I work on all these application framework things that you should check out at (mumbling) but it's not related to dependencies. Just a lot of ideas come out of developing it. So it's kind of related.

KENT: Awesome, thanks for joining us. And Ashley G. Williams should hopefully be joining us shortly as well. So we'll say hi to her when she comes up. So sweet. Let's jump into our conversation today. So both of you mentioned we are gonna be talking about. Well, we all mentioned this. We're gonna be talking about managing dependencies today. And especially npm and node ecosystem. Dependencies play a huge role. We're kind of like, in this community it's kind of a hyper-dependency community if you compare it to other communities that I'm familiar with. So, I think a first good question to kick us off though is why do we even need dependencies? Why don't we just write everything ourselves?

BEN: I think that what's great about the npm ecosystem is you can kind of concentrate on the thing you're an expert at. So Yarg is like we care about command line argument parsing and I don't care about a good camel case algorithm. By relying on a camel case dependency, you can let someone else be an expert at that. Let them write unit test for that. And it just allows you to just concentrate on a very thin area of expertise when you're writing software, which I really appreciate.

STEPHAN: For me it's simple answer. (inaudible) time or maybe ever. Because (inaudible) my own problems, there are some things like libraries that could help with them but they solve very specific problems that might be well out of the reach what I'm capable of doing given my, well time maybe but also expertise in some areas. So (mumbles) dependencies is pretty much an enabler for a lot of things that I've done. Because you can just rely on them and you know that experts, yeah, that care a lot about them and the field have implemented them. And you kind of depend on that. So that's how I see it.

KENT: So don't you think though, I agree with you. As far as the expertise stuff here is though, there are some modules out there on npm that are pretty simple, like most anybody could implement a left pad for example given enough time. So what other reasons might we want to use dependencies for these really simple functions that just about anybody could write? Or should we be doing those kinds of things? Should we maybe just copy some code? Like have a gist, have an index of all these gists that we could search and just copy code? Why do we have to use a dependency manager for simple functions like that?

STEPHAN: I think one part of that is that you think you can do it but there are a lot of edge cases. And it's really not that easy to think of all of them and to really get something correct. Because we have seen a lot of people who just forgot some edge cases, forgot some way where an undefined value somewhere could break stuff. And that's why you want to have dependencies because there is this one place where this problem is solved. And if a problem comes up, it's solved in this module and it's like available for everyone. And so you don't have to care about all these edge cases and about tests and about errors that are coming up. Because you might think it's easy to write something in a few lines, but if you look into small modules maybe, if you look at the (mumbles) scope, maybe they are handling way more edge cases than you might think. And maybe they are way more exact than what you could write just from what you had.

BEN: You know, I kind of agree with Stephan. I think the fact that you know there's another group of individuals and another community grading good unit tests and taking in bug reports is just making even a simple algorithm, making sure it runs well and it's well tested, there's a lot of benefit to that. One thing I have seen with Yarg specifically is there definitely is, especially when you're making a little utility library, you'll definitely see push back against kind of the hyper-module approach where you know you have a one line module or whatever. And I definitely do treat it as a tradeoff. I do have a lot of dependencies in a library like Yargs. But I definitely do consider pulling dependencies very long and hard. Sometimes I will just write a little utility function to avoid adding increased install time or increased depth tree. So I think it's really a balance too. I don't think everyone, I think you can take advantage of the dependencies in the npm ecosystem without necessarily going a hundred percent to the one line hyper-modular approach. There's room for stuff in between.

KENT: Yeah, I think if you're worried about a dependency adding, I don't know, like cognitive overload or being unnecessary, I think you could go take a look at what that dependency is doing and count the number of edge cases that it's handling that you're not handling. Because like often we look at a module and say, "Oh my goodness, this is like five lines of code or even it's one line of code. Why would I care about this?" But I think any developer with more than a couple months experience knows that behind just a handful of lines of code is potentially like hours and hours of debugging and like figuring out these edge cases and things. And having bugs reported and fixing those bugs. So yeah, I think that our focus shouldn't be, "Is this dependency worth it just because it's a handful of lines of code?" It should rather be like, "Is this abstraction something that I want to have included in my codebase?" Maybe it's not solving a problem in a way that I want it to and like it's going to have leaky abstractions all over my codebase. I think we should be more focused on that rather than the amount of dependencies that we have.

Now, I think though, at the same time, there are tradeoffs that you're making when you're adding dependencies to your project. Can we talk about some of the things that you're trading off by adding dependencies to a project?

BEN: Yeah, I can. One that immediately comes to mind obviously is if you start to look at the npm dependency graph, we're gonna have to make a request for each of those many dependencies. And it can start to increase install time if your graph starts looking, you know, a few hundred modules deep. So it's one that immediately comes into mind for me. But interesting, there are some interesting workarounds. Not many people know you can use this approach called bundle dependencies which is what npm itself does. And that's going to, if you do have a very dependency-heavy library, I in fact do this with NYC for a few reasons. If you have like very large dock of dependencies, you can choose to bundle dependencies and someone gets delivered your module in a single tar file. So there's much less argument against having a ton of dependencies if you're using the bundling approach. Sorry, go ahead.

KENT: Yeah, so actually, I have a video where I explain this. 'Cause I remember with left pad, everybody was like, "Oh my goodness. It's like 10 lines of code. That was so dumb. Why can't you just write that 10 lines?" And we've already talked about the tradeoffs and why you might want to have even 10 lines of code as a dependency. But if everybody had been using left pad as a bundled dependency, then nobody would have broken it. Everything would have been just fine. And so I'll link to the video in the show notes of how bundled dependencies worked. But one of the tradeoffs that I mentioned in that video that you're making is like using bundled dependencies is like locking down your version of that dependency and all of the transitive dependencies as well. Which actually is good in some ways.

So I think a lot of people had had trouble with like, "Things are totally broken, I didn't change any of my code. It just like, after this time, things started to break." And what ended up happening is one of your transitive was updated with a breaking change and now the world is on fire. And so by doing, like shrinkwrap is kind of a solution to that, but bundled dependencies is also a great solution for library authors. The tradeoff that you're making though is if there's like a security fix or a performance boost or something, you won't get that until you upgrade your dependency manually and upgrade the bundled dependency as well. And so I think though, that is a totally fine approach if you're using something like Greenkeeper. So segue to Greenkeeper. (laughs)

BEN: I just want to say. Like the downside is you can't choose pretty much. Because dependencies are always changing in a way. There will be updates, there will be new versions. Or if they don't change they might become outdated so this is also problem. So you can either depend on a very specific version of a dependency. And at some point it might become outdated, it might become problem for you to get back onto the latest version as soon as you need a new feature or a security fix or something. So you might run into problems where you have to update all your dependencies. Or if you lose version (mumbles) and say you just get all the versions in automatically, you might run into the risk of your application or your module breaking because the module might not work in a way that you expected anymore. And then your module is broken. So you can either be very specific of your dependencies and pin them all down then you have the pain of updating. Or you'll just use very loose version specifications and then you'll run into the risk of stuff breaking. So you have to find some way of navigating that dilemma pretty much.

KENT: Yeah and I don't think there's like a one solution for every project. I think you kind of have to evaluate the tradeoffs yourself. And it just kind of depends on the type of project that you're doing. Like for something that's a really small module, then maybe bundling dependencies would be a good thing. Like those dependencies of your project are part of the project that you're shipping off. But at the same time, like you are making other tradeoffs as well. Another one that I don't think we really mentioned was if you're gonna bundle this for the browser, then you could have like 40 left pads in your project because everybody's been bundling left pad. And now you have all these. With something like webpack though, it can dedupe these dependencies at bundle time which is pretty fascinating, really interesting. But yeah. I think you need to evaluate these different tradeoffs that you're making with these solutions.

STEPHAN: I think the most important differentiation there is between modules and application. Or it's the most obvious one, where when you write an application that you want to deploy into production, you want to pin down your stuff probably using a shrinkwrap file. While when you write module that other people should reuse, then you should use ranges so you don't control modules that downstream users can no longer control. And then there's stuff like CLI tools, like npm for example, where it's some sort of application but users still install it using npm. So bundling makes sense. So I think these are the three cases pretty much. And then of course there are always exceptions. But generally, I'd always use version ranges without any bundling in application tools that you install CLI tools. Bundle them and then for applications use shrinkwrap. But shrinkwrap mostly is a tool on deploy time. So you have reproducible build and like, you can get the same dependencies. But you shouldn't unnecessarily use shrinkwrap in your Git repository, but maybe we can talk about that later.

BEN: Yeah I was gonna, Isaac loves to say that npm is a bet on semver. That's a quote. So definitely, I think when you look at the things that the most work are put into in npm and where we really are continue to try to help people avoid issues they run into like left pad, definitely the workflow that we endorse is very much not locking down your dependencies for the most part. But like Stephan says, application deployment time is completely different. Like you don't want your production application to now have different dependencies than when your QA team looked at your production application. And anytime that shrinkwrap does not give you a reproducible build is considered a bug and the CLI team is working on fixing it. So shirnkwrap should, as many issues as people sometimes run into is shrinkwrap, the goal of shrinkwrap is a guaranteed reproducible build. That's kind of where we're trying to go with it.

KENT: Well, I'll add a link to the docs for shrinkwrap into the show notes, because people may not be entirely familiar with it. Actually, on that note-

BEN: I just think to say maybe you should explain (breaking up) for clarity. I can do it so. And (breaking up)

KENT: Hey Stephan. Ben, is Stephan like kind of hard to hear for you too?

BEN: He's breaking up a little bit, yeah.

KENT: Sorry about that, Stephan. Maybe Ben, do you wanna explain shrinkwrap and maybe we can figure out.

STEPHAN: Yeah and I'll try to fix in the meantime.

BEN: So shrinkwrap basically, it takes the current state of your node modules folder and creates a file that sits beside your package.json which just has a snapshot of everything that's currently installed at breakdown. So even if you're not pinning dependencies to your package JSON, you end up with a snapshot with the exact versions that are currently installed. And actually the exact registries that they were installed from. So if you were using, if some of the dependencies were coming down from a Git dependency, and some were coming down from an alternate registry you're running. Maybe you're running your own registry at work. Then shrinkwrap is just a point in time snapshot of all of that information such that you can reproduce the build exactly. You just run the command npm shrinkwrap in any project and it will create that file for you. If you're writing a utility library, I would probably not recommend publishing in that manner. Like I think Stephan was indicating, it really is more of a tool for at the time you're deploying your application to a server.

KENT: So what are the tradeoffs that I'm making by not just, why wouldn't I just commit my node modules directory to Git? That seems to me to be a pretty solid way to make sure that what I'm deploying is what I'm developing on my machines. So what are the tradeoffs that I would make if I were to just publish my, or commit my node modules directory?

BEN: Yeah, that's a good question. I definitely do lean towards bundling dependencies more so myself. I haven't used shrinkwrap too much. But I think one could argue if you wanna keep your, say that you have huge dependencies in your node modules folder that you don't want to have checked into your Git repo or published anywhere and you'd rather have those provisions on the server from the registry at install time, I could see an argument in that manner. Does anything come to mind for you Stephan?

STEPHAN: So my main problem would be Git diff. Because for me that's a very important tool in reusing and (breaking up)

BEN: You're breaking up a little bit again, Stephan, sorry. I got the gist of that. So to reiterate Stephan's point. Basically, you know, clean Git history. So every time you check in your node modules folder, you might have 300 lines of changed code showing up in your Git commit history. Whereas if you were using a shrinkwrap file, you would only see the changes in the shrinkwrap file and then the npm registry would give you that reproducible full install in your production environment.

KENT: Isn't there a way though, with Git? I'm not a Git master or anything. But isn't there a way to say like exclude this directory from diff? I don't want to see diffs for this. I know that there's a way to do that for files but I mentioned there's a way to do that for diffs. So I feel like there's probably more, like there are a couple other tradeoffs by committing your node modules directory. What else comes to mind?

BEN: I think that it's a less explicit snapshot in time, I would say. Whereas with the shrinkwrap file, you have kind of a paper trail of exactly what you have pinned things down to. Whereas with your node modules folder, especially if you have somewhat loose ranges in your package json for semantic ranges, it's gonna be definitely a less explicit pin. So you might have version 1.0.2 of a library that if you were to delete your node modules folder and reinstall everything on a different computer, you'd now end up 104. But you don't have the same paper trail that you have of a shrinkwrap file that would say, "Oh, this actually was 103." So if you check in your node modules folder, another developer deletes the node module folder and does a reinstall without something like a shrinkwrap file, they're gonna end up with a different install than you had in your computer potentially, by virtue of semantic ranges.

KENT: Yeah. Actually, that's interesting. I just saw a tweet today that said that I guess there are eight million packaged json files hosted on GitHub right now. And only 1.5 million of those are unique. And so presumably, there are a lot of people committing their node modules directory to GitHub and so that's where we're getting all the duplicate package json files. Which is kind of interesting to me. I've never actually had any project of any significance that did commit the node modules directory to Git. And so I can't really speak from experience to some of the tradeoffs that you make by doing that, but it seems to me that if you're doing that by yourself or like a project you're doing on your own, then that's totally fine and cool. And if you plan on, you don't ever plan on other people contributing to this project or whatever. It's just something you're working on on the side. But if you're working on this with a team then you can have all kinds of problems. Like some people have different versions of npm and node installed. You should probably be keeping those the same anyway. But if something isn't working on your machine, what's the first thing you do? Well, let's blow up the node modules directory and reinstall. And now you have this huge Git diff thing. Or like all these changes you have to commit with your changes. I think it's pretty safe to say it's not a good idea to commit your node modules directory.

BEN: I have another really weird kind of edge case-y reason to unbuild dependencies that I ran into at the tool I raid NYC which is a test coverage tool. I found, if you're writing a harness library that's gonna run a bunch of other people's libraries, a lot of modules use a singleton pattern, you'd be shocked. And because npm 3 dedupes modules up to the top level folder, you're gonna end up... if your utility library uses the same library as one of the dependencies that the utility library is instrumenting, then you can actually stomp on a singleton and end up having very strange bugs due to a singleton pattern. So another kind of neat trick that Scrum masters don't want you to know is that if you bundle your dependencies and you're using npm 3, it actually will push them one folder deeper in the node modules folder. So you can actually dance around... if you have a known, kind of singleton related issue, you can actually use bundle dependencies as a workaround for that. So something I bumped into before and is worth knowing about, I think.

STEPHAN: But I don't know if it's the bandwidth or the Google connection. So I just switched out (mumbling) Is it better now?

BEN: It's a little better, yeah.

STEPHAN: Yeah okay, (mumbling) So one reason that I also came up with because you were talking about teams working on things. And so you have node modules in your Git repository. You can just modify the files in there. And once that is in your, like merged in somewhere in your Git directory you might never notice. And so it's just a probability but you could actually change the module and it would work for everyone until you reinstall the next time. And it's really (mumbling). And the next thing is with next modules on next extensions. I'm not sure how again we'll handle that. Maybe I need some more about that. I think that should break as well.

BEN: Yeah, it would break. You would just need to have an extra build step on deploy time. You'd have to be running npm rebuild every time you deployed it to production. Rather than npm install, you run npm rebuild for native dependencies.

KENT: It just seems like, let's not commit our node modules directory. (laughs)

BEN: Yeah, we've come to a conclusion.

KENT: So let's talk about how do we manage these things. So we've decided, okay we're not gonna commit these. We're gonna rely on, or we're gonna put our trust in semver. And then there are two kind of use cases of dependencies that we're talking about. One is modules that we publish into the registry. Another is applications that we're deploying to our own service of some kind. And so with these two different things in mind, what are some of the other differences that we need to consider when we're managing our dependencies? What are some things that are different between applications and modules that we need to keep in consideration when we're managing dependencies? Maybe we mostly covered that with shrinkwrap.

BEN: I think the big one that comes to my mind is just like reproducibility is very, very important in applications. I mean, if I'm writing a utility library, there's this tradeoff where you don't necessarily get, not everyone's running Greenkeeper. Not everyone's going to pull patches immediately upon them being available. So it's actually a good tradeoff for me to let you have my patch releases because you may actually never upgrade to the version of my module that has that patch. So just prudence of keeping the ecosystem, having less security vulnerabilities and what have you, I think is good for shared module developers to use semantic versioning. But as we've said, if you're a company, you've gone through a QA process, it might be a day or two before it actually goes into production, you really don't want any bits changing between the QA process and your website actually being in a production environment. So locking down dependencies becomes very important.

KENT: Stephan, are you trying to talk? I can't hear you. (silence) Nothing. Okay. (laughs) Well I'll just move on then. I actually, Stephan I really wanted to ask you about Greenkeeper. How Greenkeeper works and stuff. But we'll give you a chance to figure out your microphone issues. This actually is a splendid time to ask you a question that Pam wanted to ask if she were here of Ben. So Ben, you work on the Project NYC. That's pretty much the de facto standard for code coverage these days for JavaScript's modules. So she wanted to know, a couple episodes ago, why NYC is called NYC.

BEN: Yeah, so the project kind of started out as a weekend kind of joke project. And I was hanging out at the first npm office with Isaac and Michael Rogers or request fame. And when we wanted to test, we wanted to have test coverage on npm but npm had sub-modules. So there was no way to handle, Istanbul couldn't easily handle sub processes spawning. And NYC grew out of trying to solve that use case. And so it was really, I thought we'd be able to rate it in a weekend. And it's turned into like a two year project and it's become a pretty big codebase. But at the time we were just, well Istanbul, we were trying to do something with Istanbul. There's this They Might Be Giants song called "Istanbul, not Constantinople" in which one of the lines is "even Old New York was once New Amsterdam." So basically it's a play on the name Istanbul from the They Might Be Giants song.

KENT: Nice.

BEN: And also one of the hardest to Google library names in the community which I apologize for (laughs).

KENT: Yeah we always had to see NYC coverage. (laughs) Even like NYC coverage is like you're getting news reports and stuff like that.

BEN: Yeah, that's what happens when you decide on your name as a joke. (laughs)

KENT: So actually, just out of curiosity since we're talking about it. So now Istanbul and NYC are pretty much one project now, right?

BEN: Yeah, I've been working with, I've been working to kind of merge the two projects with some of the other maintainers. And it basically came out of, NYC was getting quite a bit of popularity. But it was a completely independent project. And Istanbul was gradually moving towards a 1.0 release where it had been turned into quite a few modules from one command line tool. But it was, they needed more help to get it over the finish line. So we decided we would combine the two projects and try to get to the 1.0 release of Istanbul using NYC as the command line application for it. So we're getting pretty close. There's now an Istanbul.js.org website where you can soon read some documentation. And we're probably gonna have a 1.0 release of Istanbul in a month or so I think, I hope. So yeah, that's what's happening. We're basically combining forces to take the two projects and make a better project by combining them.

KENT: That's awesome. I think we need to see more of that. I would love to see, like this would never ever happen. But I would love to see a Browserify and webpack merging forces. Like I just think that combining similar tools to find one solution is something that I think we're getting ready for in the JavaScript ecosystem. We're still trying to figure out how to walk in this JavaScript ecosystem but we're growing up. And I think it's a good thing to see projects merging.

BEN: Yeah, that's always been my philosophy as a developer is pitch in. If there's a project you like but you're like, "Oh man, I wish this one thing was different or this other thing was different." I really believe strongly in just trying to contribute and move the whole community forward rather than trying to invent it yourself. So I think there's a lot of value to that because I mean there's 300,000 modules in the npm ecosystem. I mean I can't wait for there to be 300,000 more. We need them. But if you'd be better off contributing to one of the existing 300,000 modules, I'd rather see those 300,000 get better and better and better as well. I think there's a lot of value to that.

KENT: Totally, totally. Cool. Stephan, are you sounding okay now? I can't hear you. All right. Well I did want to talk about a couple tools. So maybe we can still work on that a little bit. So there are a couple tools that will help us manage our dependencies and one of them is Greenkeeper. And we'll let Stephan talk about that. Hopefully we get the mic stuff working. But another two tools that I like a lot are libraries.io and Dependency CI.com. So dependency is super cool. And actually libraries and Dependency CI are both by Andrew Nesbitt. And he was gonna be on the show but things just didn't work out for him to come on, so I will stumble around trying to explain these tools. But they're fantastic. So libraries.io is basically parses a bunch of registries, not just npm but the registries for all kinds of different. Here, let me just pull it up, libraries.io, for different ecosystems like Ruby and Python and Swift even and npm, Elm and Atom, Bower, tons and tons of registries as well as GitHub. And it parses through all these data and normalizes it for itself. And then it exposes a fantastic search for you to be able to search through these and get some really awesome metadata about these things. Like what are the dependencies of these projects? What are the licenses? And some really interesting things. You can subscribe to emails when things are updated. It kind of helps you keep track of the dependencies that you have, as well as projects that are dependent on your own projects. And it's not just things that are published to the registry. Anything that's on public on GitHub will be available on here. So you can see applications that use NYC for example, which is really, really cool.

So then, after building libraries.io. Andrew is still actively working on this, on this project. But he also launched a new service recently called Dependency CI that does automatic compliance testing for all the dependencies in your app. Like licenses are a really big deal and most of the time, you're probably not checking the license unless you're like a bank or something where it's like a really, really big deal. But often, I don't check the licenses as I should. And so what Dependency CI does is you stick it on one of your open source projects and I believe there's a paid tier for your private repositories. It will check all your dependencies and your transitive dependencies, like all the way down to find out if you have any license conflicts. And then you can even get a badge off of that and put it on your repo. So yeah, it's really cool. And also, I think it does a couple other things like if one of the libraries that you depend on is deprecated, like libraries.io can determine whether a library is deprecated. And if it is, it'll let you know with the badge. I think you might even be able to like send out for an email when something gets deprecated or something like that. So that's definitely something to check out. Dependency CI and Libraries.io. Two very, very cool projects by one very, very cool person who probably gets some very, very awesome contributions from very, very awesome people. (laughs) Cool, Stephan are you back?

STEPHAN: Yeah, I just tried restarting the browser. That worked.

KENT: Awesome, you are back.

BEN: We hear you, yeah.

STEPHAN: Yeah, I tried joining with the phone but that didn't work as well. So I'm now back to the MacBook microphone but it seems to work fine or good enough.

KENT: Yeah it sounds great. Cool, let's hear about Greenkeeper. We've been waiting (laughs)

STEPHAN: One thing, one thing that I wanna say about applications and modules is that you have a different responsibility, because if you add dependencies to your application, you just have to be responsible for your own thing. While when you write a module, you are responsible for what other people download and use. So you should have maybe a little bit different approach using dependencies in the module. So Greenkeeper. As I said, there's like this problem that you either get out of date once you pin your dependencies. Or you run into the problem of breaking the application by updates just getting in. So Greenkeeper tries to solve that problem by pretty much running your test speed whenever a new version comes out. So it is that you can just use version ranges and depend on dependencies very loosely. And Greenkeeper will then, in real time, whenever a new version of dependencies comes out will just create a new branch with the new version applied. Then the CI service is going to pick that up and if that breaks your, breaks with the dependency, your application or module, we'll inform you about that. And then you get a pull request. So you know that a minor or patch release just broke your application. So that's kind of the approach to solving that problem. So it gives you best of both worlds where you can use ranges to stay up to date but at least be aware of the problems at the same time. It's also just handy for keeping stuff up to date because if version updates are outside of the specified range, you'll just get a pull request as well. So if there's a new major version, you'll just get a pull request then you can see the changelog and the commits that happened. And then the CI will have run as well, and you can just upgrade and you don't have to create branches and add it to package JSON, stuff like that all menu. You'll just have it in your inbox and then you can just work from there.

KENT: So yeah when you say real time, (breaking up) like really real time? Because I've got modules that depend on other modules that I have with Greenkeeper turned on. I'll publish one and then instantly, it's really, really fast automated update. Okay, yeah, there's a new version pull request for that. Yeah, so it's super cool. This project that you have, Greenkeeper is solid. And I recommend that people take a look at that.

BEN: Yeah I was just gonna, we use it for literally everything at npm at this point. And I use it for all my open source projects. It's definitely, I'm gonna be more of a convert.

KENT: Well so. Now do you mind, actually, I'm kind of interested, maybe other people will be interested, but a couple of months ago, you had a little hiccup with Greenkeeper. Somebody was not being very nice. Do you want to talk about that? The reason that I'm bringing it up is because your response to that was one of the coolest things. I think you solved that problem really well. Like I am being ambiguous enough? (laughs)

STEPHAN: Sure, I can explain what happened there. So Greenkeeper is not a module that you use, it's a service that we're running. Because we like need to listen to what npm does and evaluate some stuff in real time. Just can't be on your local machine as a service. And we are running it trying to make the community and the ecosystem a better place. And so the thing however is that we don't have our own interface, we just use pull requests to communicate with GitHub, with maintainers and collaborators. Because as I said, we'll create a pull request for you, which means we don't have that much control over the experience that you're having. And so what we usually do is that you'll receive a pull request that is from a bot account that we've created. It's called greenkeeperio-bot. And then it will open the pull request from there.

And so and someone just created a greenkeeper account, a GitHub account and called Greenkeeper LO where the L looks pretty much like an I if you don't look closely, and just created pull requests. He copied like the title and the commit messages and the pull request body. So pretty much looked like greenkeeper pull request. But the content of the pull request was something different. Luckily, it wasn't anything malicious but some (mumbles) and they just sent it out to some very popular repositories that had greenkeeper enabled. And the problem was it just looked like the Greenkeeper pull request, so in at least one case, this malicious pull request was merged into the project. (inaudible) a little security problem there, but the problem we had is that usually, we would expect that someone will just reach out to us and talk about the problem and then we can look into it and fix it. But this was just an actual attack. Like as I said, it wasn't malicious code in the pull request but it wasn't like actual attack on the life system, to live projects.

So yeah, that was a bit hard to handle. For one, because it was on a Friday night here, but also feels a bit, yeah not right. If you want to run a service and like contribute back to the npm ecosystem, to the open source thing, trying to like run it for free for all open source repositories and then someone is attacking you in that way. So that felt, yeah it was a painful experience, like it was 2am or something. And then we had to rest somehow which was just not that cool. Yeah, and then it was horrible but we found a pretty good solution for that. Problem is not actually a problem anymore or not that big of a problem anymore because, like you explicitly enable the service for you repository. (inaudible) rights to the greenkeeper application. So we can create, like we have access to your repository and we can create branches and we can create pull requests there. But what we also can do is create labels on pull requests and statuses on pull requests.

So what we do now whenever we open a pull request, we'll add a greenkeeper status flag, so it gives you a little green check mark. And we'll also automatically add a greenkeeper label which just gives you a lot more visual differentiation. Because if you see a greenkeeper pull request, you'll immediately see the label and you'll immediately see the green checkmark and this is something that an attacker could no longer fake. And so we tried to increase the signals because as I said like it's not our interface so we tried to use the existing mechanisms to increase that. Like before, there were also some signals like our bot, which was a very weak signal because like the main, that we create the branch on the same repository while the attacker has to use the fork of the repository and then sent that in. So the name and the branch were the two signals that existed and now we add two more that are way more visible and should prevent this from being problematic in the future. But if in doubt, the easiest way would still be just to look at the diff and then see if that's actually problematic.

KENT: Yeah, awesome. I'm almost should have asked you ahead of time if you even wanted to talk about that. I apologize, I didn't, because I know that was kind of a painful experience. And not just because it was frustrating and stuff, but also, you're doing such an awesome service and it's so frustrating when people are just not being nice when you're doing something so great for the community. So thanks for doing that awesome thing for the community. Greenkeeper is awesome and you're still doing awesome stuff. And I just wanted to say that your response to that experience was super. And so thanks for managing it thing so that it made it easier for people to go (mumbles) on using Greenkeeper and (breaking up)

STEPHAN: Yeah, thank you. There are a lot of people in the community like you and the people that are actually playing nice that motivate us to do it. And so it plays out well. And so, I decided to talk about this. We didn't just introduce the security patch for that but I wrote a blog post about it. So I was pretty public about it anyway so no problems for not asking. Just because I'm not even 100% sure if the attacker was aware of what they were doing at this point. So I just wanted to, yeah point of discussion there of what your actions might cause in other people's life really. Because as I said, Friday night, it was 2 AM, the next morning, I had to get up at 6 a.m. So the whole day was wasted. It's a very little effect that you might have at your laptop hacking something but you're affecting other people. I just wanted to make that visible. Yeah, and it's worth it for, yeah, maybe getting people to reconsider their actions. But if that's not happening, for all the other lovely people that are around in this community and do cool stuff as well.

KENT: Yeah, I think like in the future, if somebody finds a vulnerability like that, rather than attacking and just demonstrating that vulnerability, something that would have been nicer would be like, "Hey, FYI, this thing is something you should probably fix." And then you could have, "Oh yeah, thanks for letting me know. I'll fix that on Monday when my life is available to do this stuff." Okay. We are running low on our time. And so is there anything else that is really important for people to know about managing dependencies before we move to Twitter questions?

BEN: I think there's, one thing that's on my mind right now is the product I'm working on at npm right now is, and it's too early to really talk too much about it but we're gonna be improving our search to try to make it so it's easier to find dependencies. And another tool I might call out in that regard is npms.io, which is one of the better search engines out there right now for finding dependencies for npm. And it ranks them based on various metrics that are very useful. So that's something on my mind. How do you actually find these dependencies. But I don't want to go too far down that rabbit hole right now.

KENT: Actually, that reminded me of something else that I wanted to mention. And that is, one thing that people get frustrated about dependencies and like having lots of them is all of the network hits that you have to make to npm, which makes npm install slower. And so I was actually having just some Twitter musings the other day where I thought of like just asking what if, almost like, what happens if the npm goes out and says hey I need these dependencies. It comes back and it says, hey okay, these are, we're gonna resolve these things and you're doing a lot of back and forth with the npm registry just to resolve versions and stuff like that. That's why I had this idea like what if all that resolution and stuff happened on the npm registry server? And then it just sent like a giant file or several files down to you with like a single download? And then you unpack that, whatever. And then poof, there's your node modules directory.

BEN: That is definitely the plan. And I know that a few of the registry engineers are, it's kind of their pet project over the next while. It's gonna be quite a bit of work to actually pull that off but that is definitely the plan. They're just trying to figure out how to use like http2. They're trying to find a way to do it with some emergent specs. But yeah, that's definitely 100% the plan. So it'd feel like, just any npm install would start feeling like you'd bundle dependencies which would feel way, way faster.

KENT: Yeah, yeah. So on those Twitter musings, Forrest actually responded and was like, "Yeah, that's basically the future." But like that would be such a major refactor of the way that things work now. I expect there's certainly no deadlines or even like an actual working prototype of it yet. So it'll be a while before we experience that glory. But, eventually that'll be cool.

STEPHAN: Yeah there are two other things I might have missed when you were discussing Dependency CI and stuff like that. But if you're handling dependencies, you have also, you have to look out for licenses if they match what you're trying to do. And you have to or should look out for security patches. And there are some cool projects around like the Node Security project or Dependency CI. So these are two other factors also in handling dependencies. And one thing I wanted to add because I didn't say it before is how to actually enable greenkeeper for you repository. Because as I said, it's a service and if you want to have it running on your repository you have to tell us about that and you can use CLI for that but I actually introduced the web interface. It's currently in beta but you can reach it at app.greenkeeper.io. And from there you can just all enable your repositories and browser and it should be really easy.

KENT: Fantastic. All right, sweet. So we do have two Twitter questions. Let's move on to those. Oh dear, I should probably have practiced this. It's Rukesh Mahapatra. Yeah, I think that's right. I think he's asking, "Greenkeeper lets us keep dependencies updated. How do we remove ones we no longer use over the development cycles as time goes on?" So from my understanding, that's not really a greenkeeper concern as much. But maybe you have something to say to that, Stephan.

STEPHAN: Just trying to, so how do we remove ones we no longer use? That's a pretty good question. I think there should be tools around that maybe because at least in your production dependencies you're probably requiring the dependencies. And so you could just look at the source code and see what you're requiring. And if that matches what's listed in the package json as well. So if modules are in your package json but not required in the code, you might want to remove them and that could be a very nice tool. That's maybe out there or not. At least I don't know it.

BEN: One thing that can help, and this only is kind of on a file by file basis, but Linters and I particularly like the Standard Project I use for a lot of my stuff. It will fail linting if you have any requires that aren't actually used in the source file that you've required the module within. And it will fail your linting. That doesn't work live. If that doesn't work, I don't know. You might have identified a library one of us should run home and write it, Stephan. Seems like a-(laughs)

STEPHAN: I'm just thinking it could be even easier if you just go in, you uninstall one dependency. Run the tests, and if it passes, you can just remove it. That will be the other way.

KENT: But that would require that I test my code, like oh. (laughs) That's great. And actually, to that note, there is an ESLint plugin that's called ESLint Plugin Imports, I think, or Import. It's fantastic. But one of the coolest things about it is that if you try to import something from a module that isn't listed in one of your dependencies, then you'll get a linting error. Which is like awesome, because lots of times, because the way the npm3 tries to push everything up to your root node modules directory, you can really easily fall into a situation where you're importing something or requiring something from a module that you don't actually depend on yourself. And so using a linter, and that plugin in particular can really help solve that problem. Sorry, that doesn't really answer that question specifically, but I'm sure that there are tools that can like look through your code and see which dependencies you have that you're not actually requiring. But I'm not familiar with any tools like that.

STEPHAN: So we maybe answered the question in a very technical way. But it maybe helps to just treat dependencies or be aware of what using dependencies means like thinking about what you do with these dependencies where removing a dependency or no longer needing to depend on a dependency should be something you become aware of. So there's two sides of that. And so if something is a mistake, then maybe you can handle it with tooling. But on the other hand, if you handle dependencies with care and think about what you do, then you should also become aware of a dependency being no longer needed.

KENT: Cool, all right. Then other question from Palmerjs. "Shrinkwrap is not fully hermentic-" which is a new word for me that means airtight or like solid I guess. "So it's not fully hermentic due to optional dependencies. What approaches have you taken to work around this?" I've never used optional dependencies, actually. Like, ever but yeah.

BEN: Yeah, I've never used, so I can't give a, I've never really used optional dependencies too much either. I've always just been told beware. But I would say that it's the goal of shrinkwrap to be a hundred percent reproducible. I don't want to speak too much for the CLI team because Forrest will be grumpy at me but I think it is the goal to make sure that if you've shrinkwrapped it, whatever you shrinkwrap will be a hundred percent reproducible. So I can't speak to any workarounds I've used because that's not part of my workflow, but I can say that any areas in shrinkwrap where it's not actually meeting its claimed functionality I think we'll be fixing over time. It definitely is the goal to have a hundred percent reproducible deployments.

STEPHAN: Do optional dependencies just show up in the shrinkwrap or is there any special handling for them? I didn't have the case for optional dependencies in shrinkwrap, so I don't know how it behaves at this point.

BEN: Yeah, I don't know.

STEPHAN: Maybe the problem is that they don't show up in the shrinkwrap, and then when you use shrinkwrap, you won't have the optional dependencies anymore. But if that is a problem, then it's not an optional dependency. So I don't know, maybe we need some more details on the question here as well. And then we can just take it further onto the chair.

KENT: Yeah, yeah. And we're definitely running low on, or we're pretty much out of time. But that happens often. (laughs) But Palmer actually did follow up with optional dependencies are used often for platform specific builds like FS events is a great example. Totally makes sense. If you want to, just go to the #jsAirQuestions to continue chatting and get follow-ups on that. I'm sure Palmer would appreciate it. Okay great. So that's our Twitter questions.

Let's go ahead and jump into our tips and picks. So I'll go ahead and go first. And then Stephan and Ben, we'll have you two go. So for me, I have two tips. Right now I am out with my family on vacation. My wife was good enough to give me some time to do this show. But my first tip is go out into nature. Life is just better when you take a break and hang out with your family and people who are most important in nature. It's fun. And second pick is play sports. I played racquetball with my wife this morning. It was a blast. We don't play sports enough together. So we're gonna do more of that 'cause it was fun.

For my picks, first I'm gonna pick Don't Break by Glove the Botmatov. It's a fantastic module. But basically it runs on you CI and if you're going to, like every single time you commit something, it's going to, well, I guess you configure on your ci, but every time you run Don't Break, it's going to pick your top ten most dependent modules or you can specify modules. And it will install all those and run those modules tests to see if the changes you made are gonna break those modules. It's kind of hard to explain. But it's super. If you've ever published something and accidentally broke somebody, this will solve that problem, pretty much. Yeah, it's awesome.

And the Next Update is one of my favorite ways to update dependencies. Next Update, you run it locally. And when you want to go through and update your dependencies, you just say, next update and it'll go through every single one of your dependencies in your package json and it will upgrade one at a time to the latest version. And run your tests, and if your test pass then it like, all right awesome. And then it'll move through the next dependency and go through all your dependencies. So it can take like a really, really long time to run. So this is something you'd run during lunch or something. But one of the cool things about it is that it actually reports success or failure to a Heroku server, and so you can get metrics on how often does this module break.

STEPHAN: So, now Kent is gone?

BEN: Oh. Maybe give him one minute.

STEPHAN: Question is, are we still on air? Like it still shows live?

BEN: Yeah. Well do you wanna, the show must go on. Do you wanna do your tips and tricks?

STEPHAN: Yeah, let's pretend we're still live and I'll still go on with my stuff. It's kind of ironic because I wanted to have the tips about two things that apparently failed today which are noise cancelling headphones. But noise cancelling is pretty awesome, especially if you're on the road a lot. It gives you a lot of focus and ability to work on stuff. So I loved them a lot. So I just wanted to say, that's a cool thing to get. And then I got a pretty good router from Netgear which just improved my Wi-Fi quality here at home and it's worth investing in your router if it doesn't fail while you're live on the show. But other than that, I'll just repeat Kent's tip. Take a break, it works, it's awesome.

And for the picks. Kent picked the two things that I wanted to pick as well. Don't Break and Next Update. They're pretty good but I have another one. It's a library called Bundled dependencies. And it will, you can run it pre-publish and then it will copy your dependencies into the bundled dependencies it will write before publishing which is what you will need if you want to bundle your dependencies. So that's a handy tool. You don't have to maintain that by hand.

KENT: Cool, sorry that I dropped off. I'm not exactly sure what happened. But thanks for picking things up, Stephan. (laughs) So I'm guessing you finished up and it's Ben's turn.

BEN: Cool. For tips, first tip is watch Stranger Things on Netflix. It was one of the best shows I've seen in a long time. It's like ET meets the X-Files. It's really fun and it's based in 1983 so it's a fun retro show. The other tip is if you wanna be a first time open source contributor and would want to contribute to some popular projects, I would love some help with Yargs which has a website at Yargs.js.org and I would love some help with Istanbul. We have like lots of first time contributor-friendly stuff. So please come and contribute to those projects.

Picks, I'm gonna say there's this library that me and my co-worker Andrew have been working on which is kind of related to a lot of stuff Stephan does with Semantic release in Greenkeeper. We have a library called Standard-Version which basically takes Angular commit conventions and generates a changelog for you and recommends what your semantic bumps should be for the package version. So check that out. If you're not quite ready to have full semantic release, but want to have a similar workflow, try out Standard Version. And then I'm sure this has been called out a million times but I love the Standard Library for my linting because it defers all of the argument about code style to a completely separate repo. And it just works, you just install it. And it means that folks contributing to your codebases know exactly what style is expected of them which I really like. That is all for me.

KENT: All right, great. Well this has been a really interesting show. I think that hopefully we've helped people out with managing dependencies. So let me just wrap some stuff up. So we do have two silver sponsors that we're grateful for, ReactJS program. They helped me master the ReactJS ecosystem. And Sentry is cross-platform crash reporting. So check them out for that. As always, we have our, I like to get suggestions and feedback. So if you go to jsair.io/suggest, that will allow you to suggest topics and guests. Like I said, you might actually, well yeah, Tweet me if you have any alternatives for Hangouts on Air. Because this show might end on September 12, if I can't find a good alternative. And then also for feedback, if you have feedback on this show or the show in general or just wanna say hi. Then jsair.io/feedback is a great place to do that. And jsair.io/email is where you can sign up for our email newsletter. So yeah, jump on that, it'll be awesome.

And then again next week we're gonna be talking live and onsite at React Rally. So that's gonna be a blast. It will be on Friday, not on Wednesday, like our normal. And the time may change. Not entirely sure exactly when it will be. But it will be great. So check out the website. Or follow us on Twitter, Facebook and Google+ to keep up with the latest. And yeah, that's it, that's our show. So thanks everybody for coming. This has been great and we'll see you all later!

BEN: Thank you!

STEPHAN: Bye!