Lodash v4 had been released (by the time this episode happens). This JavaScript library of utilities is one of the most widely used utility belts of the enormous JavaScript ecosystem. We're going to talk with John, the mastermind behind this utility (and fork of underscore) about his experience building such a ubiquitous tool and what it can do for you. We also delve into how to contribute to open source projects and how to manage them to encourage contribution. Tons of wisdom in this one!
JavaScript Air is supported by some awesome contributors.
John-David Dalton
@jdalton
Kent C. Dodds
@kentcdodds
KENT: And we're live with JavaScript Air. Hello everyone, my name is Kent C. Dodds and I'm your host for JavaScript Air. and today we're going to be talking about Lo-Dash and JavaScript details. I switched out the title of this show as JavaScript details and Lo-Dash, but I don't think the order is very relevant. So, we've got John-David Dalton with us and so we're gonna be chatting with him about that, but before we get into introductions, I just have a couple of announcements.
First, a shout out to all of our wonderful sponsors who make so many wonderful things possible for this podcast. First, our premier sponsor is Egghead.io. Egghead.io has a huge library of bite-sized videos about web development in general. Check them out for content on JavaScript, Angular, React, Node, and Elm and all kinds of really cool stuff. And then Frontend Masters is an expert-led workshop with courses on advanced JavaScript, asynchronous and functional JavaScript, as well as lots of other great courses on Frontend topics. So, check them out. And then TrackJS reports bugs in your JavaScript before your customers even notice them. And with their telemetry timeline, you'll have the context to actually fix them. Check them out and start tracking JavaScript Air today at trackjs.com. And then we have Wallaby.js. Wallaby.js is an intelligent, super-fast test runner for JavaScript that continuously runs your test. It reports code coverage and other results directly in your code editor immediately as you change your code. Check them out at wallabyjs.com. And finally, Codecov. Codecov is coverage done right. Reduce technical debt by visualizing test performance and faster code review. Codecov is highly integrated with GitHub and provides browser extensions. Learn more at codecov.io. Great, I'm getting good at saying that.
So,before we get into the show, I just wanna remind everybody who is watching live that you can tweet at the show with the hashtag #jsAirQuestion and during the show or at the end of the show, we're going to be answering your questions. And so, feel free to grill John about the concepts of JavaScript details and Lo-Dash. And then also, our show next week is going to be another live conference show, so I really appreciate any feedback you have on these live conference shows. But it's going to be at Fluent Conf on March 9th 2016 at 12:45pm PST. So, it will be a couple hours after our normal time so just be aware of that. But it will be on Wednesday. So yeah, and then as always, follow us on Twitter, Google+, and Facebook to keep up with the latest and greatest from the show. Great, now with that out of the way, let's go ahead and get into introductions. So for our panelist today, we have Brian Lonsdorf.
BRIAN: What's going on?
KENT: And Dan Abramov.
DAN: Hey there.
KENT: Awesome, and for our guest today we have John-David Dalton.
JOHN: Hi!
KENT: We're so excited to have you. John, why don't you give us little intro to yourself? And then I think you have something exciting to share with us.
JOHN: Yeah! Alright, so I'm John David Dalton. I'm the lead maintainer of Lo-Dash and the creator of Lo-Dash. I also work at Microsoft. I was the former Chakra performance program manager. And so, Chakra's the JavaScript engine that's inside of IE 9 and up, and Edge. It's also one of the, I worked to make it one of the fastest, and a lot of people have worked to make it the most ES 6 compliant too. So that's really cool. Now though, I work on open source frameworks and the libraries to improve the test coverage of them in Edge and make sure that we don't regress in Edge. So that's my day job. Outside of that, though, I do a lot of open source work, and Lo-Dash is part of that. So, cool.
So the thing that I wanted to do, though, that I was asked to do was to do a version release live on the air. Oh, so I'm gonna do it! So here it goes. I'm gonna npm-publish Lo-Dash 4.6.1. The ".1" means a patch. So here we go. Let's see.
BRIAN: Fingers crossed.
JOHN: Oh, I know. (laughs)
BRIAN: Spend the rest of the show debugging. (laughs)
DAN: Yeah, it doesn't go very well usually. (laughs)
BRIAN: Lo-Dash has historically been pretty good at non-breaking changes, so. (laughs)
JOHN: It has been pushed, and now I will tweet it out. So there it is. Rock! Okay! (laughs)
KENT: Alright npm-info Lo-Dash. Let me see this. Oh, there it is! Awesome. (laughs) Cool, yeah. 4.6.1. You saw it here first, folks. (laughs) Okay, cool. So John, why don't you go ahead and to kick off our conversation about JavaScript details and Lo-Dash, can you talk about what Lo-Dash is and give us a bit of a history of this library?
JOHN: Sure, Lo-Dash is a collection of low-level utility methods, so for iterating arrays, objects, streams, it also has methods for function composition. And I've started to branch off into more functional programming helpers, so things to help you curry things to allow composing functions easier. And that's really what it is, and I just keep iterating on it over the years.
KENT: Cool, can you give us a history of how it got started?
JOHN: Sure, I started, so I used to work for a start-up called utboo, and as part of that, I was working on a Utilities Lib for a component we were developing. And then I also did a jsconf talk on the hidden cost of Natives, which was at the time popular... it was popular to say that you would use JS built-ins, so like the array built-ins for ES 5 for performance, and that was not actually the case. There was no performance benefit to doing that, and so my talk was just going through the methods, each one, showing how JavaScript library alternatives could be faster. So, two days or four days after that talk I released Lo-Dash. Or I'm sorry, I didn't release it, I started Lo-Dash, which was a forked underscore at the time. I didn't want to recreate the wheel. I had done something similar to this before from 2008 up to that point, but it was never really released. It never got out of alpha. I did something called choosejs. I did a couple of conference talks on that too which was a drop-in replacement for Prototype JS that was faster, that didn't augment the built-ins.
BRIAN: Back in the day, PrototypeJS. (laughs)
JOHN: Yeah. So, I kind of had an experience doing the same drop-in faster alternative thing, except this time I was able to push and release and actually get it out there. And I led with a performance, because that got a bunch of eyeballs, and I actually regret doing that now because a lot of times people think, that's the only thing they think of when they think of a Lo-Dash. But I really wanted to improve cross-browser and cross-environment consistency. And I had tried to do that in Underscore first. It was unsuccessful in doing that. So then six months plus later, when I started to work on Lo-Dash, I was able to just fork and carry on from then. Shortly after that, though, custom builds became a thing. This was before modules took off, though, so I created a build tool. And then modules became a thing and then now support utility methods through modules as well. So you can just pick and choose the bits that you want instead of having to include a monolithic library. And as time goes on, utility methods increase because everyone wants a utility method, their favorite method included in the library, and we have a process for filtering that out. But over time, it grows.
And so now, I mean its 21 kilobytes. You don't want that if you only want two methods, right? So I created, I made it modular and did a lot of work to try to meet developers where they were. So, I supported AMD. There was a time where there was a big push-back against AMD use, and so libraries were dropping support. And I said, "You know what? Devs are using it. I'll support it." They were some of the biggest fans and advocates for Lo-Dash. So, great. ES6 is another one where now I support the ES6 module formats. So things like Redux, (cheers) uses it. So, yeah. That's kind of how it started. And then I never planned for it to kind of be what it is today. I've tried to just be very... to remove roadblocks for adoption because the point of the library is to get the utility methods out to as many developers as possible.
And so I don't care about your module preference. If you like AMD, if you like ES6, if you like CommonJS, if you like your arguments flipped, if you like, whatever flavor of way that you wanna consume these utility methods, but the point is just to get you high quality utility methods. So I've just tried to do that and be pretty open with the module formats and trends for functional programming and things like that, because at the end of the day, it's still just utilities. And right now, the popularity of the library, it's increasing, and so usage is going up. It's starting to chip away against the NIH or the Not Invented Here of libraries, and so that's, after four years that's... I'm starting to get adoption from libraries, which is just a dream come true. And so yeah, that's kind of how it started in its progression is just trying to get utility functions out to developers.
DAN: I've got a question. I'm not sure I completely understood. What was the original motivation behind not contributing to Underscore but creating another library? Is it that it was hard to contribute or was it technically hard, or....
JOHN: So at the time it was a substantial change and there was roadblocks from adoption from contributing before. So I had tried to fix some old browser issues before, and that didn't land. But what it did though, is one prerequisites for that patch was to make sure that the change was fast. So basically, fixing bugs but also maintaining performance. So, while that ultimately didn't make it into Underscore, the things I had to do to later on to drop the built-in use and all the other things like that, treat all arrays as dense, these were things that were fundamentally different than Underscore at the time and that were not going to make it in. It was only after they were proved out in Lo-Dash, that they were adopted later in Underscore. So it was kind of like, I was stuck road, blocked unless I forked. And so I did that. I didn't want to start from scratch, though, so Underscore was the established library at the time so that's the one I based Lo-Dash on.
KENT: Cool, so we actually got a question from Twitter that I think is relevant. So again, if you're watching live, the hashtag #jsAirQuestion is where you can interact with us during the show and we'll try to get your questions answered. But this one's from Ville M. Vainio, and he asks, "If you were doing a Lo-Dash-like library totally from scratch, what would be different? Something like Lo-Dash-FP?"
So, Lo-Dash-fp is relatively new. I mean, I've been following the uptick of FP, so, functional programming, so more functional than... that term gets thrown around a lot. Underscore is called a functional library, which because it has some composition, but this is like auto curried immutable data, data last iterate-first kind of functional programming. I don't know if I would do that first. I still like having that separation and that ability to cater to devs of either preference. It would have been things like I was locked into specific APIs because I forked Underscore and needed to drop in compatibility. But over time that has, as adoption of Lo-Dash has taken off, I'm able to break away from that and make my own API decisions and choices without having to be tethered to another project.
But starting from scratch is tough. And I tried it with, I mean I almost attempted it with that previous project, which was a DOM utility library which is a lot more complicated that like a language utility library. And so, starting from scratch is just, I would say if you can start from an established point, start there and then either try to work with them to move a project in the direction you want or if you have to, fork and go from there. But starting from scratch is tough.
KENT: If you're lucky, maybe you can do a spork like IOJS and bring it back. (laughs)
JOHN: Yeah, and we've tried that too. I don't know if that will ever fully happen. So, in June or so I started conversations for a merge of Underscore and Lo-Dash to try to get them back together. And what that actually did was that got devs from Underscore and devs from Lo-Dash to kind of hash out things that needed to be done to facilitate a merge. And so part of that that shook out was things like the four kilobyte build, where the adding utility methods doesn't scale over time. You end up getting this big library. And so one of the things they wanted was the core build that never grew. That was always just this fundamental set of functions or utilities. And so I was able to do that. They came in and Graham and Brad and Justin, they are now contributors to Lo-Dash, too. So, in that instance, the merge was to reduce duplication of effort and to help better coordinate our work. And so on that front it was a success. There are now contributors, and help move things along. (sighs) But, it's an ongoing process.
KENT: Cool, so actually on, kind of to stay on the theme of what you're talking about with contributors, one thing that just totally amazes me at Lo-Dash is I don't even have to look right now because I'm pretty positive you have zero issues and zero pull requests that are open right now.
JOHN: Now that you said that, someone's probably gonna do it.
KENT: (laughs) Yeah, probably better double check that. But if you have any there, it's just like a very small amount. Can you talk about how you're able to manage such a wildly popular project? And how much time do you spend working on this project to just be able to accomplish something so impressive?
JOHN: So, I triage bugs in the morning with my coffee. And then in the evening, I do feature work on it. So I spend about 30 or 40 minutes in the morning, and then my evening, you know, if I'm chilling out, watching TV, I've usually got a laptop open and working on a feature or something. I also do, I've started this thing where I do at least just a commit a day, and that mechanism of just a commit a day keeps things from adding up and keeps things from becoming this big pile of stuff to do on the weekend. So, I do a little bit every day so I can have a life (laughs) and do other things. I'm married. I mean like, family and other obligations. So finding that balance was tricky at first and it's been something that I've worked on over time. But a little bit in the morning, a little bit in the evening. Now that I work on open source frameworks as my day job, I can sprinkle in a little bit of triage here and there. But the key is to just do a little bit at a time, and then before you know it, you've got 100 commits under your belt in a couple weeks or whatever. So that's how I do it.
I also have started to defer more to external contribution. So a lot of times, developers will say, "Hey, can you just fix your documentation?" They'll say that of me, right? And I'll say, "Well, how about you fix the documentation? The wiki is open. You can add your own fixes to it." So, I encourage them to do that. If it's a really tiny change, I just say "Hey, let's create a pull request and get you involved with contribution and actually have you as part of the contributors of Lo-Dash." And that has helped a lot too, is just push it. And people are more than willing to help out. I try also not to be as nit-picky when it comes to accepting pull requests. A lot of times... I don't make the Style Guide a prerequisite to hit for the build to pass, because worst case, I'll go back afterwards and do a cleanup pass, right? I feel like bike shedding all of that stuff is just a barrier for contribution, so I try to be very eager with accepting a pull request once they sign a CLA if needed and then go from there. And so that has kind of helped spread the responsibility and spread the workload out. But yeah, it's just a little bit every day
KENT: I think that's just wildly impressive. That's very cool. And I think a lot of really good pro-tips in there about how to manage open source projects. So, hopefully that is helpful to people. So, you mentioned that (this is more out of curiosity and may be tangential) your work is now on libraries? Open source libraries and frameworks? And so that's part of your day job? What exactly are you doing for that?
JOHN: So, what we've been doing is things like ensuring that all of the jQuery unit tests pass in Edge, and that means either filing bugs on Edge, the browser or on jQuery's repo to fix a faulty unity test or a faulty check. But then we do that for things like Ember, Angular, and the list goes on and on and on and on. And so, usually things revolve around making sure our test coverage is up to ideally 100% passing on these JavaScript libraries and assisting when it's not, or ensuring or filing bugs in our end. I also do things like app-building exercises to shake out bugs in our platform. So last month we did a react app, which was a markdown editor and that shook out some bugs on our end with the old school document exec. command thing, which is pretty gross on contentEditable and also extended Edge support and Edge testing support to the medium plugin text editor widget. So in that case, that was a pretty big success there.
And so this month, I'm looking at Angular 1 and rxjs. And so I'll be shaking out bugs there. And so, I've got a team of two others, Zack and Jesse, and so you'll see them probably pop in to Ember or jQuery or whatever library we're focusing on and they'll be helping with bugs there too, so yeah. I mean it's also nice too because a lot of times in my off time, I focus in on utilities and I don't get to try some of the cooler things like React or Redux or Babel or, you know, the list goes on and on. And so this is now a time when I can... I don't have to just kind of work it into my unit testing for Lo-Dash, I can actually sit there and try to build an application and kind of shake out and get more familiar with these projects.
BRIAN: Are you finding some of your own utility methods more or less useful when you start to apply it to application building? Like, are you like, "Oh, I use Zip all the time now! "I never used Zip before!"
JOHN: Well, so I try to, so part of our thing is we only box it to just two days to do the app building, which is usually enough time to shake out any issues with just getting the project off the ground and then creating a minimal viable application. I do try to sneak Lo-Dash in there if I can, but usually we try to focus it primarily on the framework or library that we're looking at, and try to reduce outside factors so we can--
BRIAN: Sure, sure, but I guess in general what are some of your favorite utility methods from Lo-Dash that you find useful that many people aren't too familiar with or maybe they are.
JOHN: I can tell you, a couple of them that I thought, "Man, I'll never use this. Never." And then they became one of my favorite things. One of them was placeholders in partial application. When that feature first came across Underscore's radar, I was like, "I'm gonna have to add this to Lo-Dash now." And I was so upset. But it's become super handy for partial application. What a placeholder is, is it allows you to pass an object (in this case, it's the Underscore or Lo-Dash namespace) as the placeholder, so you can skip over that and say, it's really hard to explain this with hands but whatever. You have an argument A, you wanna skip that and provide argument B. So then whenever you execute the function, you then provide argument A, and B has already been provided for you. And so the way to signal that is to have a placeholder that is the placeholder of A and allows you to specify B. The other one that I've found super useful was the Rest function, which is similar to ES6 Rest. And Graham and Justin proposed that method, and at the time I was like, "Yeah, okay. I mean, it's in ES6." And then it became one of the most used functions in the entire library for defining all the other methods. So like, that alone, Rest is one of my favorite methods in that to work with.
BRIAN: Nice. (audio cutting out). Sorry, (audio cutting out) I'm gonna mute for a second.
JOHN: Oh, okay.
KENT: Sorry, I just muted John. Sometimes this happens in the middle of the show, that we start getting crazy feedback. So just a reminder, when you're not talking, make sure to mute yourself, and if you're on a Mac that's Command+D. If you're on Windows, it think that's CTRL+D. So, go ahead Brian.
BRIAN: Well, I had one more question since I have the mic. (laughs) Just curious, I think a lot of people admire your performance, your attention to performance and how good you are at debugging things and making sure it's very optimized. Could you just speak to your workflow of how you approach optimization?
JOHN: What I usually do is look for an established common case and make sure that common case is fast and that the external or overloaded functionality comes second. Or the extra functionality comes second. I also do it around scenarios. So I try not to obsess over micro optimizations. I would say don't focus on that. Don't worry about the direction of your while loop, don't worry about how you increment your variables, don't worry about array push versus assigning it directly to the array. None of that stuff really matters in the long run. The things that have become significant are things like avoiding linear look ups for uniquing and differencing and intersecting array, or introducing lazy evaluation and shortcut fusion to reduce tons and tons of iteration in it. And those are the kinds of things that win performance, and those are the kinds of things that developers tend to appreciate when they switch from one library to the next, they go, "Wow! My Node process is so much more improved because of things like this."
Because Lo-Dash is now used by quite a few people, I tend to get the scenarios brought to me now, but before I was just kind of having to look at the issues that were posted over and over on either Underscore or Lo-Dash's tracker to focus on pref work. But now, I make sure that the common case is fast, and then if there's not a competitive edge either, the method tends to get a lower priority. So at the moment, I have things for deep property crawling or string methods, and because there's not a direct competition where someone has called out perf on those, I tend to let those slide a bit until someone brings up like, "Hey, this thing is causing me some grief." But yeah, when I do it I don't focus on one engine. I know it's really popular to just make sure it runs fast in Node. But I make sure that what I've done works fast in Edge, in IE, in Firefox, in Safari, in Chrome, and in Node, newer and older versions of Node. I try to balance that out too. And so that is really kind of pays off because a lot of times you'll see the competition go like, "Hey we're fast in this or that." And I'll be like, "Well, that's one environment, now try the other five where it syncs performance." And it's something to keep in mind. And so I, because I also have an experience or a background with being the perfpmr engine, I kind of know the fast paths for V8, for Spider Monkey, for Chakra, and so I can navigate those and make sure that what I have ends up being great for any engine.
KENT: That's awesome. So obviously, you're not gonna be, well, I wouldn't expect that you run these tests manually in each one of these browsers and environments. What do you use to automate these benchmarks, or are you doing it manually?
JOHN: So, I do actually do it manually. I have a benchmark suite that I maintain for Lo-Dash, which makes sure that its methods are fast and I can compare them against previous versions of Lo-Dash or Underscore. And I do those before any major release. I make sure that I haven't regressed. And so usually, the Perf suite isn't there to show off. It's usually hitting my worst case scenarios, and that's what I'm checking for to see that my worst case scenarios haven't become prohibitively slow, and I check those. If I'm doing more feature work I'll create a benchmark either using jsPerf (which is having a little bit of problems at the moment), or creating my own benchmark, and then I just run it across various engines.
So some of the stuff I do is based on kind of like a heuristic. And so I pick a happy medium across the engine. So for example, I have large array optimizations where I kick in and use ES6 set or map to avoid linear lookup of large arrays. But at some point, i have to specify what a large array is. And so in my case, it's 200. Once you hit the 200 elements, that's where the cost of using setter map washes out, around other engines. Some kick in sooner, some kick in later but that's the happy medium there. So, by testing them in various engines, you kind of get a better idea of where to set those. But it's manual, but my environment I have... I used to have hundreds of browsers and versions, but I've trimmed them to just the current and then the previous one behind it on version number, except for the IEs, where I have 9, 10, 11 and so on. But yeah, that's how I do it.
KENT: Very cool. So, I think there might be some people watching right now who are in one of two camps or maybe both, well, probably not both, but people who really want to get into open source but never have or have only done a little bit and they're just not sure where to start and then other people who are project maintainers or want to start a project. And I think your experience could really provide a lot of value to those people, so could you give some tips first to the group of people who want to get into open source but just really haven't gotten around to it? What would you say to those people? how do they get involved in contributing to open source? Whether it be contributing to Lo-Dash or anything in general?
JOHN: So, usually I would say find something you're passionate about. If it's dates and times, if it's CSS, if it's, I mean there's so many areas in JavaScript or web-related stuff or even non-webby stuff to dig into. If you're passionate about something, there's probably a group somewhere or project somewhere that you could pick up and start contributing to. GitHub makes it easier with Help Wanted labels or there's a First Timers Only pull request Labels, yeah, which help allow you to pick out which ones would be a good place to start contributing to a project. I know Chakra, which was recently open sourced, has a First PR label in there. So people that are wanting to get their feet wet with some of the C++ and some of the engine stuff can jump in there. I have Help Wanted labels for my issues. I haven't started the First Timers Only, but that would be a good way to do that too. Also, I see a lot of first timers, they'll jump in on documentation, because documentation is kind of an easy place to get in, and to document something, you have to kind of dig into the source code and be familiar with it, and so that's kind of a way to force familiarity and understanding of the project is to try to actually document what something is doing.
In terms of maintainers, I would say if you're wanting to encourage first time contribution, label it and lower the barriers of contribution down. I know some people, they'll have their code style in their linters as the first things that fail, and a lot of time that's extra setup for an extra process for a first time contributor, and I tend to make those optional fails or make those things that I do on a follow-up run. And so mainly it's like saying, "Hey, thank you, thank you for contributing." So, thank the users, and then make it easy for them to contribute. And try not to bog them down at first. So, I ease contributors into the process and so after they've contributed a couple times I'll start to nit-pick style and format and things like that. Once I see that, "Oh, they're becoming a regular contributor now," I'll start to ratchet up the requirements for getting something to pass in. But I would say just be open to it. If you're running a project and you don't wanna run the project anymore, which happens, I've got a couple projects where I was no longer interested in maintaining it, so I looked for someone who wanted to help and said, "Hey, here. Here's commit rights, here's publish rights to this package so you can keep maintaining it and keeping it going along." So be open to sharing control. I made sure that with Lo-Dash, I'm not the only one that has publish rights and maintainer rights to this, so if I was to ever burn out or leave, there was other people to step in and carry on the project. So, yep.
KENT: Awesome, yeah I've actually got a follow-up question to one of the things you mentioned. So, I think it's great that you avoid nit-picking on your contributors. I think that makes it more likely that people will contribute again. So, what do you do, obviously, you want to maintain your style and you want to maintain a certain level of quality in your code. That's why we have linters. And so, what do you do, Like, you're gonna merge that pull request and then update it later? How do you do that? Does your Git history just look like a bunch of first time commits and then you fixing up along the way, or how does that really pan out?
JOHN: So, I'll either amend or do a follow-up commit depending on how bad it is. If it's just like a comma or something like that I'll pop in and do an assist. Usually commits are pretty small, and so there's not a large chunk of code that needs to be refactored. As long as the commits are small and granular, it's never really been a problem of trying to make sure things look right or are formatted correctly. I do have a style checker, though, so you can go npm run-style which then checks all of the wrappers, all of the unit tests, the entire code base for code-style rules, and then they'll flag problems. So I'll do that usually before I do a release or after a pull request that gets merged. But I tend to merge away the merge commits so it just looks like a steady stream of users contributing without a bunch of the process stuff in the way.
KENT: Cool, how do you do, sorry, I'm a project maintainer, so I've got these maybe specific questions. Hopefully they're helpful to people. How do you manage your... so what you're talking about is you wanna rebase their branch onto masters so you don't have a bunch of merge commits. But that's not possible to do with the merge button in GitHub, right? So, do you make sure that their branch is rebased first and then you can do it manually? Because if you don't do that, then you're actually creating a brand new commit and that pull request, GitHub won't recognize that pull request is merged and then you wind up closing it. Is that kind of how you do that? You just close it and say, "No, this is merged, I promise." How do you-
JOHN: No, no I make sure that they see that it's merged. So, I click the merge button, and then I go back. I pull it down. It's got the merge commit in there, and then I usually just rebase the merge message out of that and then force-push up. So, they maintain contribution rights. It shows them as the contributor, and it just shows me as a little icon next to it that says, "Hey, I assisted in cleaning it up." But they get the credit. It closes and it's tracked in GitHub, and that's how I do that. Now, that can cause some problems. Rebasing can cause some problems when people fork it and all of a sudden it's been rebased, right? And so, it could cause commits... and I've had it happen where it's like hundreds of commits and then their one change. But it's easy enough to have them rebase again. Just say, "Hey, sorry about this. You happened to do your branch or your fork right as I was merging someone else's stuff. Can you just rebase through." And if they don't know how, I'll just paste the commands that they basically copy and paste into their console and it clears things up for them, and then we go on from there. But overall, it hasn't been a big deal. Yeah.
BRIAN: I was gonna mention just for open source contribution you've made it so easy to eliminate the fear of ruining your project with the commit with a good test suite, and I think that's a huge boon for people that wanna contribute. They can just easily make sure they didn't destroy the whole thing. (laughs)
JOHN: Yeah, over the years I've tried to increase our test coverage, so at the moment we have 99.9 something test coverage. And that's, test coverage only works on a single pass, right? And so the rest of that fraction of percent is actually covered through the browser runs which are automated, and the other environment runs that are automated. So those have helped make sure that I don't break anything. I also have build tests on the CLI side, so when I do large refactoring there or adding a method or change dependencies up, it'll let me know real quick if I've broken something. But I've tried over the years to make things a little bit more fluid, a little bit more forgiving. So, people can add methods now and it doesn't break the suite or break the custom build. So, anything that's been kind of a barrier there I've tried to remove and allow more contributions.
And it's actually worked. Before, I used to have a real problem getting external contributors, but over the last year and a half or so, It's just been a steady increase in contribution and part of that was I removed a lot of the too clever JavaScript stuff where I used to have a template that was designed for HTML, and I used to have it generate the function body of a core helper function, and that was just... People would hit that and then stop, and then walk away. And so, I removed that, and then made everything just these tiny, tiny helper functions in these tiny modules, and that has just made it so much easier for people to read through the source, to figure it out. I do JSDoc-style comments, too, and I make sure that if there's something that's a little awkward in the code, I try to document that so people know exactly why I'm doing something. Oh, there's a bug in this version of the browser, that's why the code's formatted this way. So if you change that, you'll probably hit a test fail. Things like that.
KENT: Cool, I think you just gave us a whole bunch of awesome pro-tips for not only open source but coding in general. (laughs) I think that's really valuable. And I'm just gonna go ahead and give a +1 from Getify right here, because if here were here, I'm sure he would +1 that your motivation to keep the cleverness down. To make it easier for people--
BRIAN: +1 on my behalf. He's gonna pop in. (laughs)
JOHN: Oh, nice. Yeah, I just try to make it, a lot of times, I'll even do this where I'm looking at like, should I use ternary operator here? Because I can make it a string of ternary operations. Right, or just break it into an if-else. Break it into an if-else. In the long run, it's gonna be easier to understand, and so I've tried to do that. That doesn't mean that things can't get complicated, it just means that you have to document those complicated things, or have a good reason for them to be there. So, I do have tricky spots. Like, I use a bitmask, which can be something that devs haven't encountered before, but I use that to avoid passing 14 or 16 argument parameters, right? And I use that as a key component of keeping the cost of currying down. So, it's this centralized bit, but I abstract that away to just this one function, this one scary function, right? So, if you're not dealing with that one little bit, you never see it. You don't have to interface with it. It's not spread through all the other code base. It's just isolated. So, I try to isolate the ugly bits to avoid making it harder for contributions.
KENT: I think I'm gonna have to change the name of this show because we've talked about Lo-Dash, we've talked about JavaScript details, so that's been really valuable, but I think a lot of the really valuable things I'm taking away from this show are the pro-tips that we're getting about managing an open source project, contributing to an open source project and writing code. (laughs)
BRIAN: I mean, I have a question on the actual utility part, and that's really just, in my experience, I have much better luck understanding a program when my function does one thing, and this JavaScript s historically, this function takes four different arguments, does 20 different things based on your combination of type, and arguments, and things like that, and that's really to the heart of what JavaScript is in its normal utilities and things like that. So I don't have anything against that, but I'm wondering what your thoughts or experience was working with all these Swiss-army functions instead of A to B functions.
JOHN: Great question. Over the years, I've actually started to reduce overloading because overloading is a nightmare for modularity. Overloading means that you have a function that can accept multiple arguments and behave differently, so if you pass it in array, it does one thing. If you pass it in an object it does another thing. If you pass it a function as a second parameter it does an entirely new thing. Or a Boolean value. As the focus has shifted from monolithic to modular, it forces you to avoid overloading. Because people don't want to have five kilobytes of sugar for something they're never going to use. And so, with Lo-Dash 4, I have started to break out that overloaded stuff into individual methods.
So for example, unique. Unique used to take a Boolean value that would say that it was a sorted unit. And now I have a method called sorted unique. Or it used to take that same Boolean could be a function and it would use that function as a result for the criteria for the unique value. And so, now I have a function called unique-by, which then allows you to pass that function so it's not overloaded into a single piece. And so that simplifies the argument signature, which makes it easier for things like, definitely typescript definitions, it simplifies the implementation, it allows for more granular modularity and for smaller builds. And so nowadays, if i get a new method, I just try to split it out. I don't want overloading. There's some things that are stuck the way they are and I'm okay with it, like the collections method which accept an array, an object, a string, but those, I'm not growing that set. That set is pretty much set in stone. This forward is now like, let's just have an array method, let's just have an object method, let's just have a string method. And so I'm keeping overloading to a minimum and simplifying it for all the benefits I just talked about.
DAN: Yeah, I think I actually hit one of those with the Redux when I attempted to use map values. Which is simple in the simple case, but it also does a ton of stuff in the more complicated case that blows up Redux size two times.
JOHN: Yeah, this is something we address in version 4. Most of these method that accept an interatee, like a map or every or filter have called that sugar in there for shorthand. So instead of passing a function, you can pass the string like a property name or an object to perform, to create a predicate function that does a partial deep comparison. But that alone is five kilobytes. That sugar is massive, because it's doing deep partial object comparison, right? Super gross when you just want, I just wanna map some values. Like, why do I have to have five kilobytes of this? I'm never gonna use it. So a webpack gives you this really cool ability to map and remap module requires. And so, you can take that instead... there's a way to say, "Instead of using the callback helper that is in charge of shorthand to point it to a method that just returns the function. And when you do that, you cut out the five kilobytes. So, if you were to use that today, that's what I would do is I would use webpack to remap that one request to the simpler form, and then avoid the--
DAN: This is a bit like reaching out into internals, right? Because I'd have to rely on internal method names or model names.
JOHN: But Lo-Dash is developed with that in mind. So, what we do is every external function has a simplified internal function. So, if there's anything where you're having to do extra work there's always, there's usually I'll say, I won't say always, but there's usually a simplified internal function. And what we do is externally, we don't do backwards compat breaking changes until it's a major bump. For internal methods, we do the minor bump. So, all you have to do is switch your caret to a tilde and then you should be good to go.
KENT: Very cool. yeah, that's awesome. I am afraid that we're gonna have to wrap things up here. This has been really great. Thank you for coming on the show. Let's go ahead and we'll get into our tips and picks. But before we do that actually, I just wanna ask you, John, if there's anything else that you wanted to bring up.
JOHN: No, this has been great. I will say if you have any questions ping me on Twitter, ping me on GitHub. There's a Gitter chat as well. Don't be afraid to reach out. My DMs are unlocked too, so you can DM me. Anyone can do that, so I would say yeah, just reach out. I am anything. I'll sit there and if I can't help you I'll find someone else who can help you.
KENT: That's very nice of you, and a great example to us. So thank you. Cool, so let's go ahead and we'll get into our tips and picks. Brian, why don't we have you go first?
BRIAN: Sure, I just have a really small set of picks today. Should give a shout out to the PureScript online meet up, because that's back, so check that out on Meetup.com Just look for PureScript link there. And then I ran across this amazing Robot Lolita post that this guy Jesse posted and it's a really really great read. So there's that. And then continuation monad is incredible, and there's a lot of interesting uses of it with generators. I posted a link. Look at Scott Christopher's ContT's screenshots, they'll blow your mind.
KENT: Cool, thanks. Dan, why don't I have you go next?
DAN: So, I have a bunch of picks this time. And my first pick is going to be something called GraphViz and in particular, a plugin for this thing called GraphViz Review, which is an awesome way of creating, like if you want to create a flow chart with arrows between boxes, which can be of different shapes, and if you hate doing it manually you can just create a tux file in a special language and the tool is going to arrange these boxes for you. And it's cross-platform and it's pretty awesome. And this Add In plugin just boots the preview side by side so you can see how it works.
My other pick is something called WTF, which is not what you think. It's Web Tracing Framework, by Google. And this is a JavaScript framework that is like super awesome. I just played with it a little bit, and this is... it lets you boot traces in your code. For example, I'm entering this method, and I'm leaving this method, and to log events as well. And then it exports a trace that can be opened in a special Chrome or Firefox extension and you can see the frame chart, all the stack verticals. And you can zoom in, zoom out, you can look for things by filtering, and it's super awesome. I recommend it. It's great for debugging performance issues.
So, another thing is I wrote, like I said, about the flow chart. The reason I was looking for a tool to create flow chart is because I was creating a flow chart. So, I published a flow chart called React-Makes-You-Sad, which is a common sentiment I hear these days from people who just come into JavaScript programming see React, Google React boiler plate and see a bunch of projects that are super complicated and think that this is wanting you to have that you should choose React, which is not true. So I just made a flow chart that just kind of tells you "You don't have to use all of this. You probably don't need Flux, you don't need (mumbles), you don't need all these kind of fancy build tools." Because it's best to start by learning React itself, and then you can add things as necessary for your project, incrementally. So there's that.
And finally, our conversation about built-ins reminded me of an old comment by Petka Antonov, who's one of the authors of Bluebird Promise Library about how people often think that build chains are bound to be faster because they are in native code, so this is a great comment explaining why this is not exactly the case and why build chains are often slower than JavaScript implementations. That's it.
KENT: Wow, that was awesome. I totally didn't get your last pick because I'm looking at this React-Makes-You-Sad thing. (laughs) I'll have to listen to that again. That was great, I'm going to pick that, too. Very cool. (laughs) So, I have one tip. Do open source. It's great. I think I've given that as a tip before, but I just think it's great. So my picks, I have three. ReactJS program by our friend Tyler McGinnis, one of our panelists. I wish he could be here to talk about how great this is. But it's basically a training website where you can register for a course and there's a video. You follow along with the video and build an app and it's super, super cool. The first course is totally free, so go check that out. And that's ReactJS program.
And then Healthy Open Source is a blog post by, is it Miguel, I think, from the Node project. This kind of changed the way that I think about maintaining open source projects. So I really recommend that you look into it. I've already start implementing some of the ideas here, and it's already had a big impact on my project. So, very cool.
And then finally, All Contributors. All Contributors is a thing that I started just last week that basically the tagline is "Recognize all contributors, not just the ones who push code." And so, GitHub is really good at showing you, "These are the people who have committed to this project." And that's great, but there are so many people who contribute to a project who aren't pushing code. And so, the purpose of this project is to recognize those people in a prominent place in your project, because they're really providing a great deal of value to your project. So, check that out. And yeah, let's go with you, John. What do you have for us?
JOHN: I wanna comment on your All Contributors. So, I'm kind of the workforce on my project. I do the commits every day, but a large part of operations in core is just bike shedding API, figuring out the directions of features, and none of that's really code-push related. But it's a big help, and you can't run the project without it. So that's great. Non-core, non-code contributions are super important.
For my picks, though, since I didn't mention in the conversation, I'll take some of those. One of them is the Babel plugin Lo-Dash, which is our official Babel plugin for simplifying paths. So you can use Lo-Dash as a monolith, as the monolithic invocation, and then it will automatically break everything into paths for you.
The other one is ESLint plugin Lo-Dash. I like the convention there for the name. And this is not a Lo-Dash core thing, but it allows you to lint your project for specific versions of Lo-Dash. So, you can lint it for best practices for Lo-Dash Version 3, for Lo-Dash Version 4, and it helps, I've seen it help with folks migrating from one version to the next. So I would say check that out.
The other thing I wanna mention is there's a blog post called Why Using _.chain is a Mistake, it's a little bit like click-baity title, but the post is super thorough. It gets into functional programming concepts, it gets into using composition instead of chaining, and is a really good read. I would say it's probably appropriate for a Friday afternoon read, because it is a little bit longer than some blog posts, but it's super, super good.
My tip would be if you're wanting to up your JavaScript knowledge, the best way to do that, at least for me, has been to dig into a project. That's how I re-learned JavaScript was digging into Prototype, digging into JQuery, digging into new tools, digging into Underscore, digging into all these frameworks and libraries. And it just, once you dig in and you start to figure out how things work it' really helps get past, and it helps you understand these harder programming concepts in JavaScript. So I would say just dig into the source code and just start taking stuff apart. See what happens if you change a value here or a variable there, and explore.
KENT: Cool, cool, awesome. Great, so I think that is it for our show. John, thank you so much for coming on. We really appreciate you and your efforts. And with that, I think I'll just wrap us up. So, for some of our regular closing announcements: remember, if you have any suggestions for topics or guests, go to suggest.javascriptair.com to suggest shows that we should do. And then if you have feedback for this show or the show in general please go to feedback.javascriptair.com and we love feedback, so please give us some. Especially about the live conference shows, because we do have a couple more coming up and I wanna make sure that those are what you want them to be.
And then, just in general, next week is a live conference show. It's live at Fluent Conf, and really excited about this one. It's going to be March 9th at 12:45pm PST, and John is showing off his Thunder plane shirt there. Hopefully maybe we can get a live show at that conference this year too. That would be sweet. Yeah, and then remember to follow us on Twitter and Google+ and Facebook to keep up with the latest. And I think that's it. Oh, one other thing. So, if you want to join an email, a weekly email list with the show notes and things like that, we're planning on actually doing this. Go to jsair.io/email and register yourself to be on the email list. And I think that's it. So, thank you so much, John and we'll catch everybody later!
JOHN: Alright, thank you! Bye.