KENT: And we're live with JavaScript Air. Welcome, everyone. My name is Kent C. Dodds and I am your host for this JavaScript broadcast podcast that we have loved for the last 11 months or so. And unfortunately, I have an announcement to make. I am sunsetting the show. That's right. I'm actually not gonna be doing JavaScript Air after November 2nd. So sad. Sad day but sometimes there are so many awesome things in this world that you can do, and you have to prioritize them and stuff. Like, I'll just, before we get into our show, I'll explain a little bit about how this happened so even a couple months ago, I was starting to think, "Oh man. I'm just like taking on way too much stuff. And JavaScript Air is awesome and I love it but I just don't know if I have time to do it." And so I was like this close to stopping the show back in, like, August but I decided to keep going.

And then just the other day, I've been really falling behind on planning shows and stuff. My wife took the kids to a concert, and I was at home with our baby who was sleeping and so I was like, "Sweet. I can, like, wash the dishes really fast and then I can go work on this cool thing that I've been wanting to work on." And then I realized, "Oh no, I can't do that because I have to set up some stuff for JavaScript Air." And that was when I realized, "Okay, JavaScript Air, as awesome as it is, there are other things I'm more interested in doing. And now it's become a chore." And that makes me sad. So yeah, unfortunately because of that, because there are just so many other things that are awesome and I want to do, I just don't have time to do the show anymore.

So this doesn't mean that the show will never happen again. I'm gonna keep the site up and everything. You can enjoy all the podcasts, and maybe in a year or two, or even a couple months. I'll miss it and I'll start it back up again. You can hope but yeah, for now, November 2nd will be the last show, and it's going to be an awesome one. If you remember, our very first show was with Brendan Eich. We did a rerun of that just last week, I think, and it was an awesome show. And so I reached out to him and asked him, "Hey, Brendan. It would be kinda poetic if you are on for our last show, " and so he agreed to join us for our last show. It's called JavaScript and the Web Platform so we're gonna be talking about just kinda basically what the show was all about with the creator of the language. So anyway, that's a big announcement. If that's why you were watching, you can tune out now (laughs) Just kidding. (laughs) We're gonna talk about async today, and so that's gonna be awesome, so don't tune out. It's gonna be awesome.

So let's go ahead and get into what we're gonna be doing. I need to give a shout-out to sponsors before we get into the show because they are so great, and they sponsor some of the great things about the show. They've been really supportive. So first is Egghead.io, the show's premier sponsor. They have a huge library of bite-sized web development training videos. Check them out for content on JavaScript, Angular, React, Node, and even Yarn. If you're into that Yarn package manager, Jamison Dance totally made a Yarn tutorial video like today. It's awesome. Egghead.io is also the host of two free Redux courses from Dan Abramov. Find them at Egghead.io/Redux.

And then Frontend Masters is a recorded expert-led workshop with courses on advanced JavaScript, asynchronous, and functional JS as well as lots of other great courses on frontend topics. Check them out at FrontEndMasters.com.

And TrackJS reports bugs in your JavaScript before customers notice them, and with their Telemetry Timeline, you'll have the context to actually fix them. Check them out and start tracking JavaScript errors today at TrackJS.com.

And WebStorm is a powerful JavaScript IDE. Give it a try for a more productive development with ES6, Angular, and React. Use the code JAVASCRIPTAIR at checkout at jetbrains.com/webstorm to get 20% off your WebStorm personal subscription.

And Trading Technologies is looking for passionate and inventive full-stack JavaScript developers, who want to work on cutting edge solutions in a collaborative and challenging environment. Go help them build the top choice platform for derivative traders.

Alright. Sweet, so yeah, let's go ahead and introduce the. Oh actually one other quick thing. If you have questions, go with the #jsAirQuestion, and we will answer your questions at the end of the show. This includes questions about like why I'm sunsetting the show if you want some clarification or something on that, or have any questions. But like primarily this is for questions for our guests about async patterns, so good stuff. And yeah, we are going to have a show next week. I'm gonna be at Connect Tech in Atlanta, Georgia. So our show is gonna be at a little bit different time. Go to JSAir.io/calendar to subscribe to that so you get it added to your calendar and in your time zone. But yeah, it's gonna be great. Look forward to it. And yeah, another on-site conference show. Those are kinda fun. And yeah, let's go ahead and introduce the people now. First, we have Peter Lyons.

PETER: Hello!

KENT: And Val Karpov.

VAL: Hi, everyone.

KENT: Thank you both for coming. Let's get an intro to each of you before we start into our chat so Peter, why don't you go first?

PETER: Hi, so my name is Peter Lyons. I work as a consultant helping companies succeed with Node.js. And I guess the most likely channel that the JavaScript Air listeners would know me is if they've done work with Node, it's very likely that they've found some of my answers on Stack Overflow, where I am currently the number two all-time answerer for the Node.js topic. And I've got about 1,600 answers posted there. Also, I have a blog and I'm active in my local tech scene in terms of meetups, and speaking, and mentoring, and that kind of stuff.

KENT: Wow, man! Number two on Stack Overflow. That's a big deal.

PETER: Nah, formerly number one but dethroned sometime in the last year. (laughs)

VAL: Still pretty awesome.

KENT: Oh yeah. That's great. From all the people you helped, I will thank you for them. (laughs) Cool. And Val.

VAL: Hi, everyone. My name is Val Karpov. I'm the backend lead at Booster Fuels, which is a small tech startup out in the bay that does on-demand gas delivery in Texas and California. I also blog at TheCodeBarbarian.com. You may have seen me as an contributor to MongoDB open source community for Node.js, most notably the Mongoose ODM. And I'm the author of two books, Professional Angular JS, which is a guide to Angular 1, and the 80/20 Guide to ES2015 Generators from earlier this year.

KENT: Aha, generators. That is applicable to today's conversation. So yeah, today we're gonna be talking about async patterns. So I think like I always like to start things off to make sure that we're all talking about the same thing and what not. In particular, we're talking about JavaScript and async. Can we just first start out with why async is even a thing? Like in the good old Java days, there's always gonna be async like you're hitting a disk and stuff but like with Java, I would just like say, "Okay, go do this thing," and then when that thing was done, the next line would have the results of that thing. Why are things different in JavaScript? Why do we need to think about patterns for asynchronous stuff in JavaScript? Actually, sorry, let's take a step back. Let's answer that question in a second. Let's first answer the question of what is async. What are some things that we can do that results in async? What does async even mean?

PETER: So I can think of some with that quote. I think async means writing a program that can start an operation, and then continue to do other things while that operation happen in the background per se, and then proceed with the results of that operation later without doing something other than just waiting. And I guess the way to, it's kind of easier to contrast, I think with synchronous programming, which is everything you do, you implicitly wait for it to complete, and then you proceed to the next thing. And so asynchronous gives you the ability to start something, and instead of just literally waiting there doing nothing, while it's happening, you can do something else while that is processing. And async is sort of the broad category, and then different programming languages and environments have different technical mechanisms that they provide to the developer to allow that kind of behavior to happen in their programs.

VAL: Yeah. In my mind, async is kind of similar to that. It's about. (static) It's waiting for something else to happen that's not CPU-oriented like, say, a network call. You don't want the CPU to be spinning waiting for the network call to come back while it could be doing other useful work.

KENT: Oh yeah, that makes total sense so I think like, if we thought about it in what you were saying, Val, like the CPU just spinning doing nothing while we're waiting for the network call to come back, you could simulate that in JavaScript by having a while true loop, or while some variable and only set that variable when that request is finished or something like that. That would like just not make sense. So there is actually a, I think it would be really valuable for us to understand the event loop and how that works in JavaScript to like help us understand how the async stuff works in JavaScript specifically. Anybody feel comfortable like with words? (laughs)

PETER: I can certainly take a crack at it. I've explained it with words on Stack Overflow so you can find it if you search for Understanding the Event Loop in under the Node area of Stack Overflow, I have an answer there that have gotten some a lot of upvotes but basically the whole mechanism is there is a queue of work to be done and JavaScript to be run, and the environment being a browser or Node sort of takes the first little bit of JavaScript that's ready to run, executes it. And because JavaScript is single-threaded, that's the only thing that can happen at that particular moment in time and that JavaScript executes. And then while it's executing, it can queue up other things including network requests, and doing I/O, and kind of asynchronous work to be done later. You can use setTimeout. But those things don't happen immediately. They just get added to the bottom of the queue, and once the little snippet of JavaScript that's running and that's what's called the tick of the event loop, and a tick basically just means a little bit of JavaScript that can run from start to finish.

When that tick, when that JavaScript is done, there is nothing else to be done, that tick sort of completes and the event loop mechanism goes back and says, "Okay, what's the next thing that I can do?" And that list can include JavaScript that's waiting to execute because of setTimeout as well as processing results of AJAX calls like, "Oh, the server has responded to the AJAX get that you made for this resource, so I am going to fire the event callback for that AJAX request so that your JavaScript that is gonna process the response maybe take a JSON payload and update the UI." That's the next bit of code that's ready to execute so I can pull that off the queue, run it, and then go back to the top of the loop. And that whole mechanism is really all there is. It just keeps doing that until there's nothing left to do. And in the case of Node, that's how Node actually knows when it's time to exit. Once that event loop queue is drained, it goes, "Okay, program's done. I'm gonna completely stop." And that's what it does, and if there is more work to be done, it just keeps running.

VAL: Great description. I have absolutely nothing to add to that. (laughs) Way to go, Peter.

KENT: (laughs) Awesome. Yeah. So then we fire off these asynchronous operations like a setTimeout, it's the kind of canonical example because it's the example you can pull up in your browser, and not have to have any sort of database but what are some of the other operations that we do on a regular basis in a browser or a Node that do require like an async operation?

VAL: Database calls, HTTP requests, what do you call it, event emitters, things like that. (static) all these patterns kind of tie into the event loop.

PETER: Yeah, I think the browser input handling is a great example so that may be the most familiar to frontend developers is just if you add an event listener to a button, you're just saying, "When the user clicks this button, I want to run this little bit of code," and that's gonna happen later, and it may happen more than once. And so that is just there. When that click happens, that event listener is known to be listening for that so that little function that you've added as your event handler just gets executed, and when it's done, it just goes back to the event loop where then maybe the user's gonna click the button again later. So that's sort of the canonical example, or I guess in my mind, the most straightforward example. And when you get it to Node, then you don't have mouse and keyboard and touch input that way, so the backend folks think about the file system and the network as the sort of most common types of code that you need that's gonna trigger an async pattern.

KENT: Yeah, that makes a lot of sense. I think a lot of people would be familiar with that, and maybe even not realize that that's kind of an async thing. I remember I had a friend in college, and I was still getting into JavaScript and so is he. And like he really, really struggled with asynchronous code like just looking at the code like, "I'm creating this variable in this function here like why can't I use it here?" Or like, "I'm running this code like why does this code run after this one or whatever," 'cause it's inside of an asynchronous callback or something. So what are some of the challenges that you've seen with beginners in asynchronous programming, and what can you do, or what's a good way to think about it to help us when we're like reading asynchronous code?

PETER: Well, I thinK that what you just mentioned, I think, is the absolute prototypical example of-- the confusing part is if you're writing synchronous code, the order that you're coded in the file, top to bottom, is the order that it executes. So line one executes first, then line two executes second, and line three executes third, and there's something very straightforward and easy to understand about that. And as soon as you have async, line seven might not execute until after line 20. And I think that's the first real big stumbling block, and sort of mind exploder for a beginner is, "Oh, I'm defining this callback here," and inside the callback function on line 320 where I get the data out of the file but those don't happen until the bottom of my program has run, and then later on when that data's ready, it gets called. I think that's sort of the biggest leaps to first make if you've never done that kind of programming. It's just separate in your mind, "the location of the code in my file is completely separate from when that code actually run." And it might run more than once, and it might run in different orders and different instances. And once you understand that, then things start to make more sense.

And one thing I've thought about on the tooling side is our tools don't really help with this very much but I would love to see a syntax highlighting that doesn't just highlight the syntax of the JavaScript language but actually takes a callback that's in a nested function that's gonna execute in a different tick of the event loop, and change the background color of that entire code block so that you can see like all of the code that's at the left side of my file is gonna run in tick one, and then any event handler, or a callback, or a promise handler function, those are gonna be indented if your code is all formatted, and that code is gonna execute under different ticks. And if that code had a different background color, that would just make it really clear like, "this block isn't gonna happen at the same time as this other block that has a different background color."

VAL: Yeah, I find what makes the JavaScript so difficult is it really just like puts your feet to the fire in terms of concurrency from day one. Like, say you're a beginner in Python, you're all right and here you go, you're writing single-threaded code for like, say you're going through a book. You don't even look at threads until chapter 10. And if you're gonna look at, like, an event loop in Python like Tornado or something, that would be like chapter 20, if at all. So it's something where you kind of, it's really baked deep into the language. And concurrency kind of comes as a first-class citizen when you're writing Node, which is what makes it both very powerful but also very difficult.

KENT: Yeah, I agree.

PETER: I definitely remember an experience of having worked with Node and JavaScript for several years, and then having to briefly switch and do a little bit of coding in a Python project where all my code is written in synchronous style, and I was just like, "Wow, this is so straightforward now," like I can just do a bunch of stuff to the database, and then return a response, and it's very small in terms of lines of code, and it's like really straightforward. So part of me is like, "Wow, this is really great," and then there's another part of me that really agrees with, I think, what was one of my results unless your thoughts about asynchronous programming is it's really not good to hide it from the programmer. You want to make it front and center about what is code that runs right away, and what is code that runs later. And so part of me is like it's actually better to understand this code block, it could take 10 milliseconds or it could take a second if the network is misbehaving, and make it easy for you to code in a way that is more aware of the reality of what can happen.

VAL: Yeah, I guess it just comes down to your use case. I mean, if you're writing like do some analytics from the database, like yeah, Python and single-threaded makes things a hell of a lot easier. It makes a lot of sense but when you want to write a server that can handle thousands of requests concurrently, well then you're gonna have to start writing threads. And I don't know about you guys, but I find threads very confusing. I've spent more than my fair share of time debugging very strange race conditions in like Java backends, native Android apps, things like that. And I'd find that even with callbacks, it's a lot of easier to kind of reason about than having to worry about threads and locks mostly because well, the virtue of JavaScript is that code and a callback will execute sequentially. You don't really have to worry about like halfway through a block of code, they're gonna get thread interrupt and you're going to have to, "Oh, this variable can change underneath you." Those aren't necessary. It doesn't happen in JavaScript unless you have a callback and an async operation, which is pretty cool.

PETER: Yeah, one of the things I've noticed so I did a bunch of threaded Java back in the mid-2000s, and one thing I've noticed coding with callbacks and Node is, and I've never really been put to task to prove that this is true but I'm really, really confident that I can just examine visually Node-based code with callbacks and I can spot errors just right away. They just stick out like a sore thumb. And I can tell you just by looking at code without having to run it. I don't need syntax highlighting. I don't need an IDE. I can just look at callback code, and tell you if you have a mistake pretty reliably. Even when I was pretty good at multithreaded Java code, I couldn't just look at a piece of code, and see the synchronous keyword, and be really confident like, "Oh, that statement is extraneous. You can remove that and the program will behave correctly." So that's one thing I've observed about callbacks in JavaScript. With all their words, I never got to that place with threads where I have that kind of confidence. And even when I do promises, which I have done a lot lately but they're not my main thing. I don't have that same confidence. I'm always a little bit worried like, "Oh, there might be a subtle bug in here," and I find them harder to reason about just from scanning the code perspective.

VAL: Yeah. It's not just you and me. I used to work at high-frequency trading, and yeah, we would occasionally have little race conditions where it would take like a team of PhDs in the order of days to find the race condition. So yeah, threads, lots and lots of subtle, insidious bugs where it's just like, "Oh, this one little block of memory isn't quite locked the way that it should." Really, really just gets to be hairy, and I don't envy people. (static) That have to write backends in Java. (Kent laughs)

PETER: Yeah, and I find a lot of people, it's tricky to understand, and then so if you have a partial understanding what a. A pattern I saw was people just start putting asynchronous keyword there in Java just in case because it doesn't usually cause your program to break, although it may destroy your performance benefits of doing a concurrency. It may kind of kill your concurrency but that's really hard to prove in testing. So yeah, I remember very vividly. There's a file. If you have a long career in programming, I'm getting to be an older statesman here but over a 15-year career, there's a few file names, actual file names from projects that you remember. And I mean I have whole jobs where I worked seven months, and I literally forgot the entire job. I look at my resume and like I can't even remember that I've ever worked in that company but I can remember UDPListener.java that I've worked with 15, 12 years ago or something. That was another of my colleague's very first attempt at writing a UDP network server in Java, and it was just full of loops, and asynchronous keyword all over the place, and me trying to maintain that having outwritten it was really, really tough. (laughs)

KENT: Well, that's funny. I think that we can get into the same thing with JavaScript though. I've definitely dealt with race conditions, and like spent hours and hours on that kind of thing. So maybe this is a good way to segue into some of the solutions around asynchronous programming because we can say that it's easier than threads, or more straightforward, or you have more control, and so that's nice. And I agree, I really, really struggled with threads and concurrency in the short time that I used Java but there are challenges with asynchronous programming in JavaScript so I think like the form as far as patterns go for asynchronous programming in JavaScript is the callback where you have like the pyramid of doom, the triangle, when you have multiple callbacks into callbacks. Can we talk about that pyramid a little bit? Like it kind of is one of the root problems that has spawned lots of the other solutions around asynchronous programming, I think.

PETER: Sure. Yeah, I mean that's the classic, most straightforward way to write it, and then you end up in the pyramid of doom. I find the solution pretty straightforward. There's a great site called CallbackHell.com that I think Max (mumbles) originally created most of that content but it just gives you some easy patterns to get out of that pyramid of doom. But in my mind, the answer is really don't use so many closures so take all your nested functions and make them top-level functions that just do one or two asynchronous operations, give them a name, and then write the, what I call glue code that coordinates how those things execute, what is serial, what is parallel, how the values have waterfall across stuff, and keep that glue code separate from all of the actual code that's doing the work, so then you'll have a lot of name functions at the left of your file that are easy to test that only do one or two things that don't have a pyramid of doom, and then you've got your one sort of main worker function that coordinates the interaction between them. And I think both of those things make each of those bits of code easier to read even when you're looking at the glue code because those are just the names of functions. They're not the actual bodies. It's easier to see how they're being tied together and threaded together. And then when you're looking at actual bodies of each of the piece of the work, it's like, "Okay, this then clears the database for this thing and if it finds it, it does this and if it doesn't find it, it does that," and then that's the end of that function. So that's my take on the callback hell.

VAL: Great point, Peter. I wrote a simple article to pretty much like exactly what you were saying. That still ends up being one of the biggest sources of hits to my blog entitled Callback Hell is a Myth. Click bait title, definitely but my point was very much similar to what you made there where generally like examples of like really bad callback hell tend to have just one giant monolithic function that does everything. One of the examples that I went through at the blog post basically did something like, "Okay, connect to Elastic Beanstalk, put this code into Elastic Beanstalk, or whatever that Amazon thing is. Do like a bunch of file system operations, clean up everything, pop the job off," just like your function is doing too much. It's not necessarily the callback hell. It's just not building up abstractions. But what pyramid of doom really gets, or where I would always find problems with callback hell is (static) back about like four or five levels down, and your entire process crashes. So that's kind of where, in my mind, things like promises and Koa kind of come in as they really help you with that error handling, and making sure that like if you have a callback like three levels deep, you could have a common way of handling errors of that callback.

KENT: Great. I'm glad that you brought up promises and (mumbles), Val. I think we only have like about 15 minutes left in the show, and I think that it'd be great for us to talk about the various different styles or patterns. So maybe we can just list a couple of the names of those things, and then try and cover as many of those as possible during the remaining time that we have. So we've already talked about callbacks, and we talked about events, and events actually just really use callbacks unless you're using some sort of abstraction. And so I think those are kind of the fundamental concepts around asynchronous programming, and so then there are libraries, abstractions, patterns, built around this like fundamental concept that make reading that code a little easier to do and programming as well.

So can we talk about, like we've got a Google doc in here or anything. I'm not sure which of you put these in here but here are some of the major paradigms that we have in the Google doc, and we can talk about some of these in the rest of our chat. So there is streams, and promises, and generators or co-routines, async/await, observables with like RxJS, and then there is like some other things that I totally don't understand like fibers, and I'm not even sure what these things are but Zones, IcedCoffeeScript so yeah, lots of different patterns there. What are some of the things that you all see most commonly and you find most useful?

VAL: So at Booster, we're very much, or we use basically promises, co-routines, and RxJS at least on the backend. I'm not entirely sure off the top of my head what we do on the frontend but we also use co-routines and promises up there. Fibers is actually not technically an async framework because what fibers does is it actually makes network calls entirely synchronous, so it's kind of like the opposite of, "Hey, I have an async framework." The whole point is it goes into Node. It sort of monkey patches the code that makes network calls async, and then basically just makes it fully synchronous, which is useful sometimes.

KENT: Wow.

VAL: It's good for like writing like little shell tools in Node that do async operations but yeah, it has its own quirks as well.

PETER: Do you notice that Meteor is still using that?

VAL: I actually did not. I haven't looked at Meteor in a couple years, so it's been a while but yeah, not entirely sure.

PETER: Okay. My understanding of the history that's like I kind of, I find it interesting to follow the history of this stuff as best I can but I think Meteor was one of the early projects that use this fibers technique for dealing with async code in Node. And as far as I know, it's the only big project that's done so. I'm not sure if they're still using it but it seems to me like a pretty clearly, an option that's probably not gonna go mainstream, and it's probably on the way out. And just for interesting historical background, Zones is another attempt to solve this in a different way. I think it was primarily developed by Bert Belder from LoopBack and StrongLoop. He is one of the Node core contributors, and then IcedCoffeeScript.

VAL: (mumbles) Oh, I just want to say like now Angular 2 is really trying to take on Zones as like a fundamental thing from last I heard. From what I understand, Zones are like an attempt to make Node.js domains into just sort of a generic thing that you can use in all JavaScript environments. Not entirely sure that's a good idea because domains were extremely buggy and not very useful in my mind but maybe Zones are better. I haven't really tried.

PETER: Oh again, just for historical reference, I think there was something called IcedCoffeeScript, which was a dialect of CoffeeScript that had the async and await keywords, which we now are getting into the actual JavaScript itself but it was this idea that you could write synchronous-style top to bottom code using these extra keywords that they added, and they would do a bunch of transpilation to take that CoffeeScript and generate the correct JavaScript according to i. But at the early days of Node, there was a sort of explosion of people trying to solve this problem in different ways. But I think now the promises are in the language, and async and await are in the language. Now we have a bunch of stuff that is pretty clearly like, "Okay, this has been standardized." And so it seems like there is a lot of stuff that's now in the toolbox that's fairly safe to build upon.

VAL: Is async/await actually formally in the language? I haven't really checked too much but last I checked, it was still stage three in TC39, which means it's probably going to be in but on the other hand, I spent a hefty chunk of time about this time last year working on a project that use Object.observe, and well, we know what happened with that stage in (mumbles) TC39.

PETER: Yeah, I don't know off the top of my head exactly how far along. I think you're right that it's not. I don't think it was officially in ES2015, and I don't think ES2016 either.

VAL: It was definitely not in either of those but I'm sure ES2017 has been finalized.

KENT: I'm looking at it right now, and it is in the finished proposals. It's expected to be shipped in 2017.

PETER: Okay.

VAL: Awesome.

KENT: So we should be able to safely use it, (laughs) and also one of my other favorites, trailing commas in function parameter lists and calls. I like my (mumbles) for my Git stuff (laughs) but anyway, let's actually talk about the async/await for a little bit. So to really understand async/await, I think it is helpful to understand promises, and maybe in some places where promises don't like completely solve the readability problem of asynchronous programming. So yeah, maybe we should talk a little bit about how promises work, and then how async and await work to kind of make that code a little easier to read.

VAL: So promise is basically an object-oriented wrapper around a value that would be computed in the future. The core defining property of what is a promise is it has a .then function, which basically says, "Okay, once this basic operation is done, execute this function with the result if there was one, or execute this function if there was an error and pass in the error as a parameter." A nice, neat thing about the .then function is if any of the functions that you pass into .then return a promise, you can basically chain .thens on top of that, so you can hand kind of a very flat structure where you have nested callbacks (static) promise and then back at the top level, you just chain a .then on top of it.

The nice, neat thing is also there is a .catch function, which is kind of just a helper around .then with just an error callback and that basically lets you catch like errors from any of the promises in your chain. Which makes promises very neat but the limitations come in with things like-- oh so ES6 has ameliorated this to some extent but with things like looping and retrying things are not quite as easy with promises. Promise.all can help you with that which is top of all the ES6 function that basically says, "Execute all these promises in parallel. Give me there results as an array." But that still doesn't help you with things like, "Oh if this attempt, this request fails. Let me retry it after x number of seconds." That's something that's pretty tricky to do with promises right now.

PETER: Yeah, I think that's a very interesting point. For all these different paradigms, I think there is a handful of use cases and code patterns that they really shine in that are sort of the happy path for them, and then there is a whole bunch of stuff that they consider to be edge cases and become either impossible or extremely difficult to do correctly. So for any one of these patterns and some of the things you mentioned like, "Oh, I want to retry this thing. I want to cancel something while it's in flight. I want to have control over timeouts." Many of these patterns have those kind of more sophisticated behaviors are just really hard to do. And it's interesting to see, sometimes people will find that a problem, and that's where we get a new paradigm. Right now promises are not trying to cancel, and people are figuring out, "Well, how do we make these so you can start running and cancel it before it's done?"

KENT: Yeah, actually so on that note of cancelling, there is a proposal right now. It's stage one, so it's like use it at your own risk here but there is a proposal for cancel promises because like. Actually let's talk about why would you want to cancel a promise or an asynchronous scenario, like one example I can think of and that I had to implement, and it was a real pain was like an autocomplete field or search field or something like that. Somebody's typing stuff in. I wait like 300 milliseconds after the last character was typed in and then I fire off a request. And then they type another character, then I'm gonna fire off another request but I want to cancel the previous one. And just because of the way that, like how difficult, well, technically impossible it is to cancel a promise right now, when that second request or when that first request comes back, I'd have to check. "Okay, was this the most recent request that I sent? If it's not, then I have to ignore it." And so all of that logic that I had to do would be a lot easier if I could just cancel promises, which is part of the reason why there is a proposal out for that right now. But this isn't a problem with some of the other async pattern library abstraction solutions out there. The one that comes to my mind RxJS or Observables for which there is also a proposal, I believe. Anybody want to talk about Observables?

VAL: Well, so an Observable is kind of loosely similar to a Node.js stream. The general idea is you just have this thing that spits out events that you can then filter, map, et cetera, et cetera. The thing about Observables though is they try to glue like a lot of concurrency things under one umbrella, so you can actually also do things like HTTP requests with them (static) that you cancel an Observable is you unsubscribe from it, so if you have an Observable that represents an HTTP request, let's say, the Observable basically spits out bit by bit data that it gets back from the server as it gets it but once you unsubscribe, technically the Observable is, is the logger gonna receive any data? So that's kind of how cancellation works when you think about HTTP requests using Observables. But to be honest, I have never actually used Observables for HTTP or database requests. I found them useful for other applications which we could talk about if you're interested.

KENT: Yeah, I am. (laughs)

VAL: (laughs) Well, so the way that we actually use Rx on the backend at Booster is we basically have like a set of library calls that are basically just like represent our core business logic like creating a new request, or update to get existing customer's profile, or something like that. And we have an Rx wrapper around all of these things so we can basically pipe in and say like, "Okay, every time a customer does anything, we want to log it here." And Rx makes it very easy to do that just because every time one of these events finishes, our system basically shoves that out into an RxJS stream that says like, "Okay, the model name is customer. The function name is this." Or it's like update or whatever, and we have the ability to just like filter through the stream and do whatever we want so things like fraud detection, Slack notifications, logging, debugging, just printing debug messages, all of that goes through Rx.

KENT: Cool. Actually one thing that I think would be valuable for us to talk about with each of these paradigms, abstractions, whatever you want to call them is in like what cases are these not suited for? Like in what situation would I not use Observables? I know that the mantra is "everything is a stream" but is everything really a stream? Are there situations where Observables aren't really a good fit?

VAL: If you're not comfortable with Observables, it's the obvious one. (laughs) Observables, because they try to fit a lot of things under one umbrella, they are very confusing, and it becomes more difficult to kind of think about what's going on with an Observable. Sometimes it's easier to just say like, "Okay, it's a promise. If it fails, I'll retry it, and if I need to cancel it, well, I don't need to cancel it so."

KENT: Yeah. I have yet to jump into the Observables thing. It's sound really cool but I'm not sure that I have found a use case for it, or I just don't know. Maybe I'm missing out big time. What are some scenarios where-- we kinda talked about where promises fall off for a little bit but yeah, actually let's just. We never really gotten into async/await. Can we talk about why async/await is a thing? Like why are promises not good enough? And why did we create this async/await thing, and maybe how does it work a little bit?

VAL: I think it all just comes down to being able to use like for loops, while loops, and try/catch to handle async code. I think like async/await and its kind of predecessor, Koa and yield are. What they let you do is basically pretend that async operations are synchronous-like so as long as you put await something, you can wrap that in a for loop, y ou can put it in a try/catch block to catch any errors. It makes (mumbles) like a lot easier because you're writing async code, and it is actually fully asynchronous. It doesn't block the CPU or anything but it looks like synchronous code and you can use the same old for loops, return statements, try/catch blocks that you're used to from Java but actually get the benefits of the async code.

KENT: Yeah, it seems pretty cool. I wasn't actually totally sold on async/await when my coworker decided we should try it out in our code base. And so I was like, "All right, let's try it out. We'll see." It took me a while to warm up to it but just the other day, I was working on something, and I was like, "Okay, I want to make this API asynchronous. I want to enable callback or whatever." And so making that API asynchronous with just raw promises would have required a pretty significant refactor but using async/await was just like, (laughs) I was really shocked to just like, "Okay, this function is async so just put the async keyword there, and then put await in front of this other thing that it's gonna like return a value later, returns a promise ultimately," and like I was done. And I didn't have to do any weird reformatting or refactoring at all. I was really impressed, and that's when I was sold. "Okay, yeah, there are some times where async/await is pretty awesome."

PETER: Yeah, I thInk the interesting thing for me is it has a stack trace plays in all into this. I feel like there is a big part of the JavaScript language that is asynchronous first, and when you have an event loop, it turns out that the stack trace is not that useful because the event loop constantly starts new, small, very small, shallow stack traces, and there is no mechanism to link from when an exception occurs on one tick of the event loop, they don't normally know that on some previous ticks, that was the actual cause of why the code got into a bad state but that the JavaScript does have try/catch and does have stack traces, and so when you write synchronous code and you get an exception, you have a stack trace, which is sort of an error handling or a debugging mechanism that's built into so many languages and it is quite useful. And then once you start doing asynchronous code, it sort of gets taken away from you, and you have to use different tools to debug things. And then with async and await, try/catch tries to stitch things back together even though the code has become asynchronous and has done asynchronous operations, and there have been different ticks in the event loops with different stacks, they can try to weave those back together into what they call a long stack trace but that's like, "Oh, tick 17, you made this database call and then on tick 42, it came back with an error, and then the code that was supposed to do with that error had a bug, and it caused a TypeError and race exception." There is your long stack trace.

VAL: Yeah, definitely--

PETER: In my mind. In my mind, it's just like I always detest this. It's not clear. I'm like, "Do we like stack traces? "Are these good or do we like event loops? Are these good?" And like, I don't feel like we've found a way to sort of have our cake and eat it too.

VAL: Yeah, that's one of my favorite features of async/await is the stack trace. I guess right now with ES6 or ES2016, I guess right now you can do most of what async/await does using a library like Koa and the yield keyword and generators but what you don't get is the stack trace. So you get the similar benefits of being able to do for loops, and try/catches and while loops, and use return statements in async code but you don't get the stack trace benefits. You still just get, "Okay, process stuff next tick at the top of your stack trace." Limiting sometimes and it makes it confusing to debug but for the most part, I have been able to work around that. There is also libraries like the npm module longjohn actually likes to sort of monkey patches things to give you long stack traces pretty easily in Node.

KENT: Yeah, we love our monkey patching in JavaScript. That's for sure. (laughs) Yeah, that and fibers, right? So yeah. And sorry, I was just joking. I'm sure longjohn is terrific. (laughs) So we just have a couple minutes left in our show, and we didn't even talk about generators. We should probably bring those up. Some people really, really love them, other people really, really don't, and many people really have no idea what they are, how they work. Does anybody want to take a stab at explaining what generators and where they're valuable?

VAL: Yes please. I just wrote a book on it (laughs) so I have a lot of things to say about that. (laughs) So the way that actually, last I checked, the way that Babel and Traceurs transpilers for async/await work, they actually compile down into generators because what a generator fundamentally is a function that allows re-entry. So normally, when you return from a function, it's done. That function is gone. It's lost all of its state. Everything is finished. With a generator, you have a yield keyword which lets you say, "Okay, partially return this value but retain the state of the function as long as a reference to the function is still living," and then you could call .next on the reference to the function to resume execution whenever you want. It can resume execution asynchronously, which is why we just have libraries like Koa work. Fundamentally, you basically yield an asynchronous primitive like say, a promise or a thunk which is a function that (static) and then a library like Koa, or spawn, or Vue, or whichever you prefer basically waits for that operation to resolve and then calls next on the function to resume execution at a later tick but the actual business logic within the generator function is entirely flat. It's try/catches. It's for loops. It's all of those nice little primitives that you learn when you're beginning JavaScript.

KENT: Cool. Yeah. I think that sums it up so what are some scenarios where you would use a generator over async/await?

VAL: So we like to use it just for like the backend business logic, and just sequences of HTTP request or database calls. Where it really doesn't work well is streams of events like button presses or things like that on the frontend. Usually on the frontend, you kind of want to use something like Rx, or mob, or Redux. That is probably geared towards that. I think like generators and async/await are just really geared toward just functions that kind of execute once and terminate after a while, as opposed to something that does like a while true effectively like waiting for button presses to come in. It is technically possible. I actually wrote a blog post about how you can handle like a Node.js stream in Koa, or effectively using async/await. It's pretty gnarly, and it uses the other promise.race function but it's technically doable. It just looks really ugly. It feels very like Java-y (Kent laughs) because you have like a while true loop and you have a promise.race to see what event comes in first, which is a little bit messy.

KENT: Could you explain promise.race for those who don't know?

VAL: Promise.race is kind of like promise.all. It executes all or a bunch of promises in parallel and returns the first promise to resolve or the first error that occurred. So yeah, it gives you a promise that resolves to the first promise that resolved in the array.

KENT: Perfect. Cool, cool. Well, that's all the time that we have. We didn't get any questions on Twitter, so we had a little bit more extra time. If anybody does have questions, we'll go through our tips and picks. I'll keep Twitter open. So with the #jsAirQuestion, you can ask questions about async patterns or why JavaScript Air is over or whatever you want.

Okay, great. So we'll go into our tips and picks. I'll go ahead and start. I have three picks. First is Yarn. It's a new package manager client, not a new registry or anything, still using the npm-registry and everything. Npm is totally in love with Yarn. It's all great. We're all happy. No forking of the community but Yarn is a really impressive package manager from some really impressive people so check it out. That link will be in the show notes, and then repl.it. I think I may have picked this before but it's a really impressive site that actually one of our past guest is the CEO of it, I think but it basically gives you a repl in a browser for a bunch of different environments, JavaScript, Python, like Ruby, a whole bunch, and it's a great teaching tool so check that out. And then relevant to today's show is from Getify, Kyle Simpson. It's called A-Tale-Of-Three-Lists that implements the same thing in a bunch of different async patterns so check that out.

And then yeah, just a tip, a general tip I'd give is like make the choices that you want to make. Like do the things that you think are the right thing to do, and don't worry about what people are gonna say about you, or what other people think. Like, if you think that it's the right thing to do, and that's what you want to do, then do it. It's part of the reason why I decided it's okay for me to stop JavaScript Air. As much as I love to please people, and I want to be of use in the community, there are just like, I didn't think that it was the right thing for me, and it wasn't something that I wanted to do to continue doing it, I guess. And maybe I'll come back and do it in the future. So anyway, that is my tips and picks. Peter, why don't we have you do yours?

PETER: Sure, so my first tip is seeking out mentors and peers. So make sure you have some peers in your career that are at sort of similar place that you are that you can bounce all ideas off of, and then also mentors who are maybe at a place that you're trying to get to. And it doesn't matter what you're experience level is so I've been a professional programmer for 16 or 17 years now, and as soon as we're done with this call, I am meeting with my mentor to review my goals for next year, and try to learn from him, and get his advice. So do that. It doesn't matter. It's not just for beginners.

Second tip is try to cultivate a self-awareness so when the tools that you're using are either not visual enough or not fast enough. I think in programming, it's a lot of right connection of file, the machine disappears for a while, and then you get some result that may or may not be expected. And if you're doing frontend browser development, you have some decent tools to debug and see trends but a lot of times, just the tools you have don't show you the time aspect that you need to see or the right data in a convenient format. So as soon as you get that sense, start writing a script, start writing a little web UI, some kind of dashboard, something that will give you the right information without too much noise with the right amount of signal, and it will do it fast. And that's one of these like level up. This is how your productivity can sort of make a leap and bound. It's like you spent half a day, or a day, or two days writing a tool, and then for the rest of the six months, that tool is like a superpower for you so do that. So those are my two tips. Are we doing picks at the same time as tips?

KENT: Yep.

PETER: Okay, so I have two picks. The first one is a site called Regex101. It's Regex101.com. If you're doing regular expressions, it's a fantastic tool to learn and develop and test. And it's one of these things that gives you immediate visual feedback with really rich in information, and trying to do regex development by changing some code, and running a script, and looking at the output is really slow but putting a bunch of stuff into Regex101.com, you'll get the correct regex really fast, then you'll just see how it fails and how it works. So that's my first pick.

And my second one is the blog of Julia Evans, which is jvns.ca. She is @b0rk on Twitter. That's B-0-R-K, and she has a fantastic blog where she goes into synchronous programming, and a lot of deep topics like networking and containers, and the Linux kernel, and system calls. And she just has a great style that's very inspiring to show that you can dive into really, really deep topics, and you can learn them, and just great writing style and great technical information. So those are my picks.

And then my random pick is the sport of bouldering, which is a flavor of rock climbing, and I've seen in my Twitter feed it's mostly other developers, and I've seen more and more people getting into it. I think it's a sport that a lot of developer types can enjoy 'cause it has this sort of puzzle solving aspect to it but it's a great way to do a physical activity. So if you haven't tried it, and there's a new invite. Go and try it. You can usually get a day pass for pretty cheap, and that's what I do for my physical activity these days.

KENT: Awesome. Thank you. Val.

VAL: Cool, so my tips, first one lines up kind of with what Kent said earlier is just kind of think things through for yourself and do things that make sense for you. Again, just because like some expert on Reddit says that you should be using Observables, or async await, or Koa and yield doesn't necessarily means that's the right fit for you or in your team. Just kind of sit down and figure out what you're willing to invest in and what really solves problems for you.

Number two, I really encourage everyone to kind of write about things they learn, and write about stuff that they're working on. Either like give a talk to your team, put up write-up in Slack, or even write a blog post. There's always like the best way to like solidify your learnings about something is to try to teach somebody else even if they don't listen. (laughs)

And a couple of recommendations, if you're interested in learning more about like generators or promises, and promises are very important because they underlie both co-routines and async/await so you should probably be very familiar with them. Pony Foo has a great series called ES6 in Depth. It's PonyFoo.com. Look for ES6 in Depth. You'll be able to find it (mumbles) really great detailed guide to think ES6 concepts like generators, arrow functions, and promises which are just very fundamental part of how we write JavaScript these days.

And random pick, I just got myself back at (mumbles) for coffee a couple of weeks ago. And man, productivity through the roof, workout's through the roof. Highly recommended so check that out if you haven't.

KENT: All right, great. (laughs) Okay, so that's our show. Let's just close up with a couple of closing announcement. I just want to give a shout-out to our silver sponsors that we're so grateful for. ReactJS Program, they help you master React and React Native. Find them at Jsair.io/reactjs-program, and thanks to Hired.com, they bring Java offers to you which is like the coolest catchphrase ever. Find them at jsair.io/hired. And yeah, I've actually heard really good things about them. I've never used them before but I heard good things so check them out. And then a couple more links for you. I was actually just about to say jsair.io/suggest but since the show is wrapping up, and we have all the shows out. This is kinda sad. I'm not going to recommend you do that. You will be disappointed. But I still am interested in your feedback, and if you just wanted to say, "Thanks for the show," JSAir.io/feedback is a great place to do that. And then JSAipr.io/email to look at our past newsletters. We do still have a couple more issues coming out so yeah, check that out there. And then again, we have a show next week. It's gonna be on-site and live at connect-js. It's not gonna be on like same time, same place. It is actually going to be at 12:30 p.m. Eastern Time on Friday on the 21st so be prepared for that, and with that, I think that we're all done so thanks to both Peter and Val for coming on the show. I really appreciate it.

PETER: Thanks, Kent.

VAL: It was great to be here, and yeah, best of luck with the rest of your projects, Kent.

KENT: Thank you. I appreciate it. Bye!