Energy Modeling: the Good, the Bad, and the Misleading

Chris Keefer  0:00  

Welcome back to the couple. Today I'm joined by a returning guest, Mark Nelson for a very special episode a highly anticipated episode, an episode that the listenership have actually been asking for on and off the record. And that is an episode on modeling, energy modeling. And really, you know, I'm the son of a couple of academics, and I was a bit of a

Speaker 1  0:25  

rebel without a cause in terms of ignoring some of the great gifts of knowledge. But this word epistemology, the theory of knowing how we know things, I think, is what we're going to be unpacking today. Because modeling is, it's obviously all around us. It's very influential. You know, I run an advocacy organization that's poor and unable to kind of get into the modeling game to compete on the battleground of ideas with modeling. And I'm wanting to understand it better. So who better to have back then our consultant here, and the couple are some people have made the mistake calling the co host of time, times, but it's been a little too long, Mark, thank you for coming on to help desk this one off and take us behind the curtain to understand modeling better.

Mark Nelson  1:17  

Of course, Chris, and of course better to be confused as your co host than as literally you. I mean, I don't mind being confused as you that's a great honor for me. But you should, you should be careful, because you might lose your medical license.

Chris Keefer  1:31  

All right, all right, Mark. We don't even do the self introduction stuff. But I'm always curious. You know, where in the world is Carmen Sandiego. You're constantly globe trotting. Just take rather than a self introduction, just take a minute to tell us about your most recent journey to give us a bit of a timestamp, despite the Evergreen nature of this topic.

Mark Nelson  1:50  

Sure, I spent last week in a Estonia, one of the most fascinating places I've ever visited a real blank spot on my map during a youth spent, I guess, early early adulthood spent going to Russia and studying in St. Petersburg, and, and Central Asia, in fact, had several journeys across former Soviet Eastern Europe and I just had never been to the Baltics. But when finally because of nuclear, and the Estonian nuclear program, getting going, I made a made an excuse to go see it, I was stunned by what I saw, which is basically a hyper efficient little city state enclave by the sea, a city with a very proud history tolerant as one of the great trading capitals of Europe, the sort of gateway to the howling Eastern wilderness of Siberia, and the trade goods that came out of that vast land, but also the gateway to the west, the early factories, early mechanized cloth industries in the low countries and the wool networks in Britain. So Thailand has been this cosmopolitan trading hub of the North for 1000 years, kind of, and it's fascinating to think of that as being the place that may bring truly bring small nuclear, to the forefront. And the reason why I like small nuclear for Estonia is because it's literally a small country, they've got just over a million people, about half in the talent capital area. So if they say SM Rs, I'm listening, what I don't get as much enthusiasm for is claims about SMRs from giant countries that should be capable of programs of arbitrary size, and, and scope. But in Estonia, you get the feeling that getting 300 megawatt SMRs there would be a Goldilocks situation just right. So I was there for the Fermi or an inorganic conference, always a great thrill. I don't I don't work for or even with Fermi, I'm just a big fan of their effort and their audacity to try to start up private development of a nuclear program that could be key to the flourishing or even survival of them of the Estonian nation.

Chris Keefer  4:08  

Well, you know, we are going to be expanding on on this topic and many others in an upcoming episode with you called the state of the atom. And it will take a kind of broad swath and survey of nuclear around the world. And so you've given us a little sneak peek on the SMR side of things and on the eastern European, geographical or Baltic lokaal. But we'll nip it in the Bible there because we've got big things to carry on towards. So Mark, again, talking about modeling. I find myself you know, engaging with a number of political leaders. I'll often particularly on the the left side of the political spectrum, here claims made that the the evidence is in the proof is in you know, we can obviously do 100% You know, wind, water and solar, you know, that inevitably leads to marks each because then there's other obviously other modeling efforts occurring out there. Now,

Mark Nelson  5:05  

can we just slow down for one moment? I know you're going somewhere, but you just said, the big MZ J, one of the most legendary towering figures in energy modeling, I think we should briefly explain to our audience who that is because I still even though he's a big name in our lives, there's a lot of people who don't know this gentleman. Sure. Sure. So Mark, Z Jacobson, I'm not exactly sure what the Z stands for. But it's such a good letter that you don't need to understand it to appreciate its its beauty. So Mark Jacobson is a professor at Stanford, like many of the most outstanding figures in energy modeling, he does not have an engineering background. In fact, that may be an important part of being a good energy modeler, and we'll talk about that later. But Mark Jacobson was an atmospheric sciences professor, who drew I guess, modeling weather and atmosphere stuff became more and more involved in the clean energy world. And he doesn't like nuclear. Why, who can even tell you he just doesn't like it. So his passion is modeling. Zero Carbon systems for not just cities or states, but entire nations or even the whole world, based on nothing but wind, water, and solar. So water power, being hydro, he has no aversion to hydro of the most profound scale, as we'll get to in just a moment. So Professor Jacobsen is at Stanford. Now, Stanford is one of the most prestigious engineering schools in the US that has no nuclear engineering. And one can start to get a little suspicious. And whenever I see that I'm speaking with are drawn against Stanford grads on subjects of energy, I kind of sigh and say, here we go, again, because you can typically assume they don't even know if they know what they're talking about in their area. They don't have exposure to or really an appreciation for things nuclear. And I think that comes from institutional qualities to Well, Mark Jacobson is a perfect example of the Beyond almost all others of a Stanford professor, ultra prestigious position, title, instant credibility with press all around the world, whenever he comes out to make a statement, and they're always anti nuclear, and he became famous beyond the world of energy modeling, because various Hollywood celebrities, working in the clean energy space decided that he was the guy that was telling them what they wanted to hear that you could have a world where everything was cheaper, cleaner, easy, no nuclear, no fossil fuels. It's an intoxicating vision if you're a neurotic, anti Nuke, and you want to show that there's some reason to plunge towards renewables with nothing else. So he became the energy guru to say Mark Ruffalo, and to Leonardo DiCaprio and their foundations, and he became the guy for Bernie Sanders. Now, why would Bernie Sanders want to work with a guy like Mark Jacobson? Well, if you're in a state like Bernie's, where 80% of your electricity comes from a single nuclear reactor, turning that off, would seem to blow a pretty profound hole in your energy system. And it's, since Vermont effectively doesn't allow renewables because they don't like them. They're, he's hard to understand, like what you do next. But if you can show that there's a Stanford professor who can demonstrate that Vermont is going to be just fine losing what 80% of its electricity production, and still be able to be part of a low carbon world, then that's, that's a seductive view. That's a really beautiful view. Of course, it was bullshit. They just use natural gas power from other states. But the claim is good enough to execute sudden political plans. So listeners waiting to hear which side I fall on energy modeling.

There's one of my first cards on the table, the use of various tools, energy modeling, including them to execute sudden, large political changes. I'm quite weary of that. I mean, I would like if it's towards a direction that I intuitively like, but most of the time, nowadays, I see it as a dangerous thing. So what's this tool this Mark Jacobson became very famous beyond even the fame of the the energy modeling world and then the celebrity clean energy world. He became famous when I believe it was 2018. I need to check on that where he published he had published a paper one of his big landmark papers that that the Hollywood types loved which said you can power the USA on wind water solar only. And he didn't include access to his code. So we haven't even properly defined modeling here, but let's do it in order to explain Why it's interesting that he hadn't provided us code and how he got into a crisis that made him even more famous, we might say that modeling is the use of mathematical equations to represent the world in order to understand it. Now, let's narrow it. It's the use of mathematical equations to represent part of the world or something in the world in order to understand it. And then if we say energy modeling, it's the use of mathematical equations to represent energy production and consumption, in order to understand it. And then finally, as we've seen in the case of activist Professor Jacobson, in order to change it, in order to influence it in order to dine with Hollywood, shall we say, and presidential candidates. So Mark Jacobson's paper said that you could decarbonize the US for cheap with wind, water, solar. And he said, you wouldn't need to make any extra dams, you could just use the dams that were already there. But something very strange happened in his paper. In his paper announcing his findings, and the results of his code results of his model, which was called load match, he just showed that all the consumption than us hour per hour historically could be matched by historical weather patterns harvested and transmitted around the country. Now, this is definitely a Copper plate model. One of the early technical terms we're going to introduce today, what is a Copper plate model, copper plate model assumes that say the whole area under study is a big conducting copper sheet where you can send any quantity of electricity produced anywhere at some time, to anywhere else at that same time. In other words, the US is made of not one grid, but three grids, there's limited transfer capacity at the moment, the copper plate model of Jacobson and others assumes that that's no trouble, you just build what you need to transfer the power and just assume it can make it there. And there's no downed power lines, there's no transmission disturbances, there's no load limits on power lines, there are no apparently it's it's just anything produced, can be sent anywhere, you can send power from Montana, to New York City if you need to. Now in the real world, there are constraints even from New York City to just over the border in New Jersey, something like there are actual constraints and getting a large amount of power to an extremely dense area, when that area does not have much power generation itself or does not allow power lines to be built. That's an issue but not in the copper plate world. You could say, No, this is a simplification, but it allows us to at least calculate how you would do it. Well, how you would do it. Maybe that ends up being severely limited by the transfer capacities while not in Mark Jacobson's world. But everyone knows copperplate models are just a thing you do if you just want to go fast and break things and get published out. But there's something else. He showed a graph of just an arbitrary couple of days, it's not clear how he decided to choose those days and his model to just to make a graph a beautiful squiggly lines that went in the paper and showed something's happening here. It's very serious. And the model is doing interesting work going up and down and balancing society's needs with the production of the of the wind water solar system. But there was a strange thing, Chris, because people who looked at this picture saw that the line representing hydropower in the US went up to 11 or 12x, the total installed capacity of all hydroelectric dams in the entire country today, at the same time, four hours at a time. So the argument, by the way, is that if you as long as you match up the total historical hydro production over the course of a year, you can have as much as you need of it from this model at any given point. And once a bunch of researchers found this and started talking to each other and said, Wow, this is totally bogus. This guy gets all the Hollywood stars, but his model is doing something that is physically impossible. They reached out to him and he said, No, my assumption is that you can just add turbans add turbans to the dams. In other words, let's say you have Hoover Dam, right, it's the bottom layer is filled with terabytes. There's a certain amount of height drop on the water, there's a certain amount of water flow that's tolerable downstream, certain amount of water available in the reservoir upstream, what he was assuming or not assuming it was probably a mistake, but I don't want him to sue me. Don't sue me, Mark. He claims he assumed you could add unlimited numbers of turbans to all the dams in America so that they could have 10 or 12x. Their maximum full floodgate everything open, you know, full rush of water, and you could have 1011 12 times that for hours at a time. So we're talking mass destruction events like go Laser dam breaks sort of changing events downstream if you were to run physical water at that rate, but his model didn't run physical water, there was no water really in his model. There were no low reservoirs in his model. There were no timing limits on when you could and can operate a dam because of animals or humans working downstream. There was no limit to how much how many turbans you can add, in this space of a dam 10x determines and Hoover Dam Why not there's only a narrow little canyon there. And then he said, It's my assumption. Now, this was all in the framework of saying you could cheaply make a national energy system on wind, water, solar. So what happened? The scholars talking to him about saying this is this is a huge error, he wouldn't admit error. He was said there was an assumption. They said an assumption of that size as first of all physically implausible. And you didn't write it out anywhere on your on your study, and he said, Well, maybe I will next time anyway, the fight got vicious. Mark told them that he would not change his position, they decided to publish as a big pack of researchers in the the, what was it PLOS One, one of the open source science journals. Then mark, Professor Jacobson sued the journal and sued one of the authors of the paper. One the only one without a research institution backing him up, sued them for damages, demanding retraction and punitive damages. And the lawsuit got leaked. We won't talk about how that happened. But I was I was there to see some of it as it unrolled hundreds of people were viewing the leaked lawsuit and writing about it reporters and MIT Tech review was one of the early stories, reviewing a big fight over modeling. And then within a few days, the pressure got so immense, that Professor Jacobson retracted his lawsuit and said, See, I made my point. I you know, I can let it sit. Now he did get countersued and the damages are currently being worked out. But the point and of course permanently damaged Jacobson's reputation. As a young reporter, it takes even a second to look up like who he is, is going to have to not probably use his his work anymore. doesn't really stop the Hollywood guys. But they'll they'll get bored and move on if they if their attention changes. What we have here now is that the professor, modelled something that was physically impossible. It got headlines around the world, it changed the terms of the debate. It's still influential today. And he did it while teaching large numbers of students. Okay, so there's the Mark Z Jacobson story. I know you didn't want to maybe spend too much time on that sort of clownish outcome of energy modeling gone wrong, because almost all energy modeling is radically more sophisticated than that. I guess the question is, what is that sophistication? What is it getting us? And does it get us out of the problems that were so clearly on display there? A big bold headline from a model that was clearly broken, the headlines take hold and go around the world faster than the modeling can be refuted. Other modelers, perhaps being rightfully resentful of the attention and money headed towards this gentleman, because of his fast, crazy copperplate modeling, it's easy enough for a quick computer to set up and run because it's so unsophisticated. And with such broken physical assumptions that reveal that the group doing it is in complete disconnect from the from the real world. But all the people that are complaining here, they have modeling, too. And I'd like to get further into my experience working with some of the folks with the more sophisticated models.

Speaker 1  19:01  

Well, absolutely. But before you do, you know, what I was trying to set up, I guess, in that introduction was this idea that there's, there's different ways of knowing obviously, right? Without the budget, you know, for my organization to hire modelers were often stuck with with examining experiments, either contemporaneous or in the past. And I think, you know, as a nuclear advocate in Ontario, we have some great history to draw upon of a very rapid build out of Canada nuclear in our, in our case, 22 reactors commissioned in 22 years 15% of our national electricity 60% electricity of our province, and we can try and point to policymakers and say, hey, look, you know, we can do it not to be too cheesy. We've done it before. Let's Let's examine what the historical conditions were. There's limits to looking at things in that light. Obviously, the future is dynamic technologies are changing. There's different politics at play. So this is also an imperfect way of knowing and being able to predict, but it's I think, has the benefit of not incorporating outlandish assumptions, which are just physically impossible in a way that models can. And so I my bias is to be skeptical towards modeling. I think, you know, another sort of epistemology question or theory of knowing that's formative for me in terms of my, you know, academic training and background as a physician is that, you know, just as I was beginning my medical training, we really underwent a major paradigm shift from a mechanistic model of of illness of have the burden of proof to show if a medical intervention works or not, it used to be well, I have this molecule that binds this receptor. And we know, you know, in terms of the cell biology that this will trigger X, Y, and Zed reaction, so the medication works, you know, around the early 2000s, we moved into this paradigm called Evidence based medicine, where those claims needed to be tested. And with an attempt to eliminate you know, every bias possible in epidemiologic studies called randomized controlled trials. And so that's taking an idea and and exposing it to a real world test. So these are kind of my biases, and now applying, you know, my interest in energy and energy transition, and trying to understand what are the kind of analytical tools we have at our disposal to imagine these worlds and to convince policymakers about the viability of certain visions? I think that's that's the question what sort of brings me to wanting to decode this, you've given us kind of, perhaps the strawman story or or kind of a caricature of modeling done very, very poorly, I want to make sure that we, you know, engage with more sophisticated modeling. And they're not just sort of, you know, being accused maybe of taking cheap shots. So let's, let's, let's go a little bit deeper, then let's, let's unpack this a little bit further. And, you know, we've got the Marty Jacobson started the way it's a really important one, because as I was mentioning earlier on, certainly, you know, I think a lot of responsible modelers say, you know, models are not to predict that or to better understand, you know, the variables that we're playing with, but certainly

Mark Nelson  21:59  

a follow on, follow on for the professor Jacobs story. I'm one of the only pronuclear people he hasn't blocked on Twitter. One, I'm extremely polite and deferential to my my superiors in the academic world, I am a PhD dropout, or even kickout, depending on which way you want to spin it. And I greatly respect people who are able to stick around and finish a PhD, even when it may not be clear what you're headed towards. So I'm very nice to Professor Jacobson on Twitter, and he stays cordial enough to me. And so I was in a front row seat to see something astonishing after the Texas blackouts. So Texas had a fairly brutal long hold spell of a very low wind. Now, that didn't change the natural gas portion of Dr. Jacobson's models because he doesn't have natural gas, it's when water solar, he jumped in after that event and said, aha, my solver solves for this too. I can show you that it would take a mirror and he didn't put a price tag on it. But at a even generous pricing several trillion dollars worth of batteries to easily solve easily and cheaply solve this for this exact situation happening again. He openly said right there in public and then defended that his study, which he put out only a few weeks after the the blackout, the devastating blackout. This is the one in February 2021 Winter stone. arry made a lot of hot takes on on energy, Twitter, but Professor Jacobson's take was that if you just installed several terawatts of batteries, terawatts, we have gigawatts today, so that's 1000 times greater, so three orders of magnitude more, you can easily ride through easily and cheaply right through with wind, water, solar and the Texas blackouts. What to me the sad is that he's entered a level of conservatism. That is that shows that his models have been altered since the time that we found the the whole back in the hydro going 12 or 13x. For him, he doesn't really see the difference between a multitrillion dollar plan and a non multi trillion dollar plan. It's all cheap and easy wind water solar to him. So that almost made it made it clear that he he is able to change. It's just updating your model after a brutal severe catastrophe of the previous way of thinking can lead to results where you keep saying the same narrative. It's easy, and it's cheap, but your own model no longer believes it. Now a bunch of other energy modelers like the ones we're going to talk to like less the, you might say that more sophisticated ones said, Oh, this is ridiculous. You wouldn't need that much. But they also didn't completely phase out fossil fuels. So it was an interesting moment. I wanted to ask you something. Normally, you're the one asking questions. I'm just granting this this paradigm shift and medicine. Does this mean there's room for things that show big evidence of working that you don't have any model for how it works?

Speaker 1  25:01  

Yeah, potentially, for sure. I mean, generally speaking, you're gonna have, you know, lower tiers of evidence, you know, things like observations that are made in the field anecdote. And those are used to generate for their hypothesis. But the ultimate crucible, the test they must pass is, you know, the cluster randomized control trial. And so, you know, that again, that's that's kind of the threshold that that that generally has to be passed for, for an treatment to be accepted nowadays.

Mark Nelson  25:30  

So even a sophisticated model that shows why things should work is it's been downgraded in importance compared to checking to see whether it appears to actually work. Yeah, but of course, we rarely have population wide experiments in public health. I mean, the COVID vaccine and booster shots were unusual. And being one of these, where you had a truly population wide check that he, of course did have a large amount of studies paid for with a large amount of money very rapidly, right off the bat in order to make what still according to the data appears to be at quite a positive at this point, public health intervention. Of course, it's a messy subject. But if we're comparing to the grid, we have something a little trickier, where it's not clear exactly how much backwards you can go or how how much you lock yourself into a certain path that you may decide on or embark on with modeling, where even if the modeling starts to change and say, Ah, something's occurring, we didn't realize this factor. When we model down to lower levels of the distribution grid, not just the transmission grid, it looks like we have this these extra costs that suddenly have to be spent. And that messes up the money we would need for that part. And then this part doesn't work like so I'm just, I'm not saying I don't trust the most sophisticated of the decarbonisation through wind, water, solar and a few other technologies, storage models. And we'll talk about some of the most sophisticated ones that I do admire, and I do think are fascinating experiments that have, in some ways proven me wrong, and my expectation is only five or six years ago. But if you're if you're wrong, and things are not going well, and this non traditional transformative way of producing and delivering electricity, it's not clear if there's an effective way to cover to a configuration that works and may hopefully even be low carbon. Whereas with the historical situation, you can call that the equivalent say, of randomized controlled trial, where we have historical examples of large scale decarbonisation that is cheap over long periods of time, through nuclear. Now, we must admit to ourselves that if this is our standard of evidence, maybe we would not have been comfortable with the extreme rapid build out of experimental technology, like large light water reactors, becoming central to the survival of an energy system of an entire nation like France, or as we saw this year, when France is sick, when France has nuclear fleet is not doing well. And then goodness is starting to recover. But when it's down, it causes problems for the rest of Europe who realize suddenly how much they rely on France, not messing up the nuclear fleet. So we ourselves with this skepticism, we have to admit, we might not have trusted modeling that showed that you could decarbonize France with nuclear energy. Now, having said that, let me give a quick answer to partly partly derived from some very interesting thoughts from Professor Jesse Jenkins, one of the most famous and respected names in energy modeling internationally that he gave in a podcast a few years ago with David Roberts. He said that in the old days, because fuel based technology was essentially fungible, you can burn this at this time, or you can split this uranium fuel at that time. And just you can assume that it's available, when you turn it on, it meant the time factor was suddenly not important. And that energy modeling often did not even have time as a variable except to say, do we can we physically hit peak with some combination of patent plants? And what are the economics of how high that peak is versus what plants we build over which period of time, but the hour to hour, just say nothing minute to minute, or for the love of God second to second modeling that we need to get closer and closer to when we have highly shifting flows from highly shifting production and a renewables world? It wasn't there at all. Because it was reasonable to believe that you didn't need it. If you could reasonably assume you could turn on essentially all of the power plants that you built for your system. That is the assumption that upon going away required a level of sophistication that as own The increased? Well, this raises

Speaker 1  30:01  

the question of, you know, what gets modeled. There's a quote that's attributed to a variety of different people. I thought it was Benjamin Franklin, but apparently not as AJ Liebling, freedom of the press belongs to those who own one. You know, there is a convergence of, you know, modeling studies, which tend to favor a certain distribution of resources. You know, Jenkins, classically, it's this, it's this construction of, you know, very low marginal cost, variable weather dependent resources, right, which when the when the weather is cooperating are incredibly cheap. It's what he calls clean, firm power, not baseload, but clean, firm power, and fast burst. And so in most of these modeling studies, we saw Jacobson, there's a very strong ideological motivation to keep it to the WWE is the wind, water, solar. I think in Jenkins work he plays around with Well, let's see, if we add a certain amount of, you know, x to the grid, how do we get to our lowest cost options?

Chris Keefer  31:04  

I think you were involved in discussing some some modeling work with a Bloomberg entity. And, you know, just it gets us to the question of what does get modeled, like my organization, we thought about, you know, hiring a modeling firm to look at, okay, what would a clean energy transition in Canada look like? If it was, you know, driven by nuclear, we don't have the funds to pay for that study. We thought maybe this would be a valuable thing to commission as a source of evidence to provide policymakers certainly, you know, the David Suzuki Foundation, one of our big environmental NGOs, has has, you know, hired modelers to, to come up with sort of their visions, there's a degree of kind of motivated reasoning motivated modeling that occurs. So in terms of what,

Mark Nelson  31:46  

yeah, I have a few anecdotes that might shed some light on this money and modeling. Sure. And we can probably discuss the interpretation of Professor Jenkins work later. I feel I feel especially fascinated by Jesse and his career, because we both served similar roles for some of the same people spread apart by several years. So I kind of seen as the guy who came before me and took a different intellectual path towards understanding the energy system, shall I say, we were both at Breakthrough Institute and then both worked for Michael Shellenberger for a number of years. And so it was this weird, parallel thing where I'm fascinated by his works, read almost all of his papers, follow his tweets, you know, it's, it may be a one sided fascination, but it's been very interesting. And it also keeps me from being in danger of misrepresenting his work, which I actually do see a lot. I want to defend a little bit, I see a lot of misrepresentation. I can say mean things. But let me just say, the nice stuff first. So let's put that aside and talk money in modeling. anecdotes. You mentioned a thing I told you about gentleman, coming from one of the Bloomberg Philanthropies so at environmental progress out in Berkeley, we had visitors from time to time and if Michael was out, traveling the world saving nuclear plants somewhere in some far flung destination, it'd be saved me at the office to answer the door when I wasn't traveling with him. So I answered the door one day, and a gentleman came in. And he was an energy economist, you know, reasonably high ranking guy who'd worked with a lot of famous organizations and had been hired to go around to get support for some pathways being looked at by a Bloomberg Philanthropy group called Beyond carbon, I think, is what it was called, it was nearing launch, they wanted to sort of soft launch and tell a bunch of people about it. So he comes up to the office, and he puts down modeling work the top line findings, and I bring up my my technical assistant, to come look at it with me. And we look and immediately I see some problems. They chose four scenarios to present to policymakers and also little people like me. So first of all, whatever modeling was there, however good or bad, it was done. They distilled it into four scenarios to present. And of course, Chris, there are so many different technologies, who can even work on the number of you know, the millions or billions of combinations of different technology selections to model, but what did they choose? I noticed that they showed that a cheaper outcome was a low nuclear high renewables electric vehicles scenario. And it was much cheaper than the higher nuclear hydrogen vehicle scenario. And I said, Excuse me, I have a little problem with this hydrogen is typically looked at as a as a fuel because it has storage cap capacities, that could sort of firm up renewables with extremely heavy thermodynamic losses, of course, but it could kind of firm up renewables. If you built out a hydrogen distribution system. You could maybe store hydrogen in the summer from the winter time, renewables or vice versa, but you don't need IID hydrogen cars if you have the baseload nuclear, and he looks at and he said, Well, this is just the combination, we chose to illustrate the pathways. And I said yes, but your, your illustration is you put a much more expensive, much less developed technology that engineers have a major problem with, because hydrogen is a wreck. Hydrogen is a mess. I, it gives me a really bad feeling, Chris hydrogen is just an ugly thing. Maybe necessary in some cases, but you have to get over massive engineering barriers to use a nasty gas like that. But they put it in with the nuclear in such a way that that may alone have been the thing that made nuclear more expensive or worse than the low nuclear scenario. And then the claim was, oh, no, this is just our way of showing some of this scenario. That's some weak shit. That's, that's real weak. That's just not. And I and I kind of pointed it out. And I said, No, we're not moving on until I understand why you put the hydrogen cars with the nuclear scenario in order to go around the country and present this to policymakers. And it couldn't end it turned a little nasty. And we had to completely switch the subject. And we had a better conversation after that. And we did not refer to the modeling again. Now, again, like with Professor Jacobs, and I don't want to put us at a risk of clowning all modeling by showing these ridiculous situations. But that second one, we're getting a little closer to the mainstream, to mainstream money to mainstream influence, and to the mainstream ways of turning what may be just fine modeling into a political presentation, that ends up having very large pathways effect if it gets into law. Okay, so that's one, the Bloomberg thing. Here's another one, I sat in on a few sessions, wrapping up the review portion of the MIT futures of nuclear study back in 2016 1718. And so there at MIT, I saw some folks coming in from I think, Dr. Jenkins was still involved in MIT at the time of some of this energy modeling effort. And they had labeling on scenarios where they call the scenario, the high renewables scenario, and they had another one that was like low renewables, high nuclear anyway, labeling that has survived, in some forms into Professor Jenkins current work with Princeton, which again, I see the Princeton stuff as the leading American modeling. For better or worse, it's the lead in America, it's not directly anti nuclear, it's not destined to say that nuclear is bad. In fact, they themselves go out of their way to point out the role that nuclear has to play, and especially in edge cases in their modeling, right, the previous version of this was being used. And it was presented to us the review board of the futures of nuclear study, which included representatives from some of the organizations who had paid for this, which included pro nuclear foundations. So the results are being presented to us in this form. See, nuclear is good, it helps lower the cost and is great, but I noticed something weird, the labeling on the model said, high nuclear scenario, and the percentage of nuclear in there was a percentage that is small by historical standards of high nuclear nations. Whereas in the high renewable scenario, the percentages were radically large compared to anything that's ever been done in the history of the earth. And I pointed that out, and I asked, I feel bad, I shouldn't pick on grad students, I was a, you know, I was a dumb grad student 1.2. But I'm very smart. And you know, math, and that the most sophisticated I ever was at holding a very large amount of ideas in my head at any one time and putting code together and presenting complex subjects and reading papers in detail. But, but I didn't know anything. And so maybe this was picking on the grandson, I asked him, Do you know, historically, like what percentage of nuclear power counts is high? And maybe he was just flustered and wasn't. But he could couldn't answer pretty much anything. And I realized that I'm sure he could go and google it on his phone later. But he hadn't. He didn't know he didn't actually have a historical idea of what should be considered high engineering or low engineering. Now, if you just answered no, by higher low engineering, or sorry, by higher low nuclear, what we meant was our expectation for our nation and what it could feasibly get done by this period of time with this amount of industrial but what that would have been a little better, but instead, I got an answer along the lines of, well, our funding comes from XYZ group and that they're not as interested in nuclear. Alright, let's move on to one more thing. One that's perhaps a lot more personal to me, where I felt most like an outsider because of my very simple modeling and logic processes on a very real political issue. And how almost everybody that now I come to see as like allies, or I work with or they're on the same side, and most of the arguments that I get into in the public at the time, they were not just enemies, they were attempting to destroy us. Now, my name wasn't super visible on this work. So I was it was a one sided taking it personally back in environmental progress. But what subjects and I'm talking about the great Mopr debate that's minimum opposite off the price rule, M O P r, which was a rule that people thought it was something that people thought was going to come down and effectively from the Trump administration in 2017 and 18. The idea was, the Trump administration would do something to intervene to mess around with electricity markets, to stop coal and nuclear plants from closing under the assault of persistently cheap natural gas, and increasingly present renewables.

Speaker 1  40:43  

And was part of the rationale of this just an energy security question of, you know, here's plants with large amounts of energy that you can store on site that are less vulnerable to this sort of just in time nature of natural gas. Is that what was driving it? Or was it?

Mark Nelson  40:55  

Chris, this was before the Texas blackout? All the modelers updated their assumptions after that, right? No, no, this was you Sure. From the Trump side, it was sold as this is about energy security is about protecting America is like keeping coal plant workers and jobs. And of course, the, you know, the smart folks, academics, the decarbonize users, the renewables, folks are like coal is dead, it's dying on its own. Not that we love gas, but gas is killing it. And if nuclear dies to that just shows that and then the pro nuclear folks, which was much smaller group in 2017, than it is now especially in the mainstream environmental groups. They were basically looking at this and saying this protecting coal, which we see as wrong, is that worth it to save nuclear? Everybody but essentially, US Michael Shellenberger myself, maybe there's a few others don't want to offend anybody. They all said, anything that protects nuclear is not worth it, if it protects coal. And Dr. Jenkins, I have to say was on the other side of this, we're basically saying, if you have a rule that protects the net that protects coal against the natural loss of coal plants, through age through being uneconomical versus natural gas, and renewables, then Soviet and if you have to save coal to save nuclear, don't do it. And where it got really nasty is some of these groups that again, I countless friends now started citing, first of all attacking us directly for standing up for the nuclear plants, even if it was saved some coal plants and they produce this big study. One of the big analytics firms did it one of the famous ones energy modeling firms used by utilities used by regulators used by nonprofits, one of the biggest firms wish I remember the name, and they had this study that said that the carbon cost of saving nuclear is not worth it, if it saves coal, and all these pro nuclear organizations mainly advanced nuclear, but not heritage nuclear, but whatever. They said, See, this proves that Shellenberger and of course Nelson was there, but it did. I my name wasn't really present. It's he's, he's totally wrong. He's not being sophisticated, his numbers are wrong. He's totally off base here. And then I started looking through this stuff being trumpeted by the pro nuclear, decarbonisation orgs. And the modeling that showed that it wasn't worth it to keep the nuclear plants if a kept coal, assume that not a single nuclear plant in the country got a license to operate beyond 60 years. And in fact, it assume that a number of them close it at just beyond 40. In other words, you cut off more than half the life of a nuclear plant in the US in order to show that saving, it wouldn't save on carbon. Plus, then it gets to one of the big things one of my big obsessions and my issues with modeling, which is asymmetric payoffs, ah symmetric risks, or even the fact that in the real world, there are no do overs. But in academia, there's unlimited do overs, the lack of skin in the game in modeling the lack of skin in the game thinking you don't, you don't say, Oh, this model didn't turn out this professors fired that doesn't. That didn't even happen to Dr. Jacobson, for God's sake. Right. So you had this situation where the modelers think let's set up a clockwork contraption, a set of models, or sorry, a set of mathematical equations, a set of you can represent in different ways a bunch of nodes, a graph, we have a situation here, we set the rules at the beginning, we snap our fingers spend a lot of computer time, it spits out the results, but the real world is constantly updating in the real world, there are constant changes in the real world, there might be a spike in gas prices. In the real world, we might allow nuclear plants to extend beyond what the modelers claim. In the real world. A coal plant saved by a Trump subsidy today can close tomorrow, but a nuclear plant that closes today is not going to be there tomorrow. So there's a fundamental asymmetric payoff from temporarily moment. entirely saving both coal and nuclear, because the nuclear can go on to survive decades, decades, decades decades. And the coal plant can easily just be shut down tomorrow hacking can be bought by a philanthropist, entrepreneur renewables gasp person and be closed to get an environmental payoff or something like the coal plants can still be closed, the nuclear plants cannot be revived. Therefore, it is wrong to have fought that particular battle the way they did especially wrong to have accused us of being pro fossil fuels or biased in order to support it. That was a major as the kids would say, Red Pill moment for me, when I realized that very sophisticated people with PhDs putting hundreds of 1000s of dollars into models were just straight up wrong in very obvious ways that they themselves detail in their methodology for anybody to see. And yet we were wrong. We were the outsiders, we were the extremists. And we barely had to do any modeling to show what was wrong with their work. Anyway, we lost a bunch of nuclear plants, there was no thing that saved them. And by the time that the Ukrainian war shut off of Russian gas, the build out of LNG, export facilities and the the whiplash from COVID. By the time all that happened, and now all the nuclear plants in the US are profitable, magic, right? It's too late for the nuclear plants. We lost then. But the modelers said it was okay, even though the world has changed beyond what they thought when they ran their model. So that that bugs me a little, the modelers can go back and update the world has to struggle on, that makes me really cautious and careful. And at the minimum, we've got to read the freaking documentation, we've got to read the manual to these models. And if the model if the manuals written poorly, you just in my opinion, toss out the model. Now there is modeling out there with astonishing detail, astonishing quantity of documentation, we'll get to that later. But just wanted to put those stories out there as ones that illustrated for me the danger of modeling, and the danger of modelers, the more sophisticated people get on the modeling, often, the less they have room in their little brains for real facts on the ground. And you can miss some of the most basic things imaginable.

Chris Keefer  47:20  

Yeah, I mean, it's stepping back from this. You know, just thematically, you get the sense that there's a lot of people working on on this modeling. It's abstracted, it's divorced from from the real world, potentially from real world engineering experience, or transmission distribution experience, these kinds of these challenges that come up, as you're saying, you can make a mistake and just rerun the model, without really paying any direct consequences. And last year model becomes policy. And, and then there's going to be real world impacts. But you're not the intellectual author per se of that the policymaker is, but there's this kind of maybe godlike arrogance that can come I think, from playing with these these bits of code. You know, I'm just thinking about prefer the godlike

Mark Nelson  48:03  

arrogance that comes from looking at France's nuclear build out.

Speaker 1  48:07  

Fair, fair. But it's some some things that I'm thinking about here, which which expose that kind of disconnect from the real world, or how the real world works was, it was on that Dr. Volts podcast, which I really do recommend people listen to that episode of the Jesse Jenkins, I think it's a great illustration of his thinking and a great introduction to modeling, and how it's done. But, you know, one of the things he brought up was, you know, a critique of wind and solar is that, you know, jobs potentially aren't as good. So we just modeled, you know, paying wind and solar workers significantly more money to get them up to a level maybe in line with a just transition for fossil fuel workers. And, you know, that is a lovely idea. And what it showed was that, you know, labor costs were in a large proportion. So it was, you know, affordable within within the modeling, confines. But that's simply not how the real world works in terms of labor rights, in terms of, you know, how workers struggle and gain just working conditions or gain, you know, wage strikes or maintain the wages that they have. And there's been excellent journalism, by advice, in particular, looking at solar installers and just what a sort of dystopian world that is for those folks, by nature of, you know, having very transient jobs that are de localized, etc. So that was just an example that kind of got my goat, of the ways in which I think when you're operating in this world, when you have, you know, all of the respect of these institutions of these politicians, when you have, you know, a lot of funding, and you're playing around with these variables, you can get this kind of god complex. And for me, that kind of just rubs me the wrong way. I'm not sure if someone react just

Mark Nelson  49:35  

holds, by the way, Chris, whether or not the system would actually hold together physically. In fact, it may be even worse if you correctly demonstrate that you can have levels of renewables beyond what skeptics and Fuddy duddies, like honestly me a few years ago would have thought possible. One of the interesting conversations I had in Estonia is with the GIA Tachi executive you just asked me straight up, you heard that I was on podcasts and he misinterpreted that as me being a podcaster. I said, No, not me. I don't even listen to him that one would Jesse Jenkins was pretty much the first and I still haven't finished it. But he asked me straight up. What were you wrong? Most strong about, right? You go and talk to people in public? What are you most wrong about compared to the past? And here's this and it gives me a chance to bring up one of my favorite modeling adjacent rules or anecdotes, which is from Jesse Jenkins himself, before he delved into levels of sophistication beyond those of us mortals outside of academia, he had what I now call the Jenkins rule, a rule of thumb that said that when the percentage of power on the grid over the course of the year, rises higher for a given energy source, then that energy sources, average capacity factor over the year, costs escalate dramatically. I'm gonna slow that down when, when the amount of wind on a grid say is higher than the capacity factor when on that grid costs mount. And so that's a really, it's not, that's not modeling. You can model stuff around it. In fact, some of the most famous energy modelers and professors on this topic had work on that in 2013, and 14, like Dr. Leon Hirth. Over in Germany, he had a very famous paper that those of us in the pro nuclear community pounced all over that showed that costs escalated for wind or for solar, when the penetration, the amount of wind or solar on the grid got to more than that wind and solar's average capacity factor, right. So this is a very beautiful, simple rule that I perhaps pushed a little bit further and just assumed that that would mean that you couldn't or that you wouldn't install. Beyond that amount of wind or solar on the grid. Or if you did, there would be some kind of short term, not medium term, not long term, short term catastrophe. I was wrong. And Australia is fascinating here. I talked to Simon Holmes, a court sort of a bad boy for a lot of folks in the nuclear industry. But he's he's very interesting. And he's got a lot of perspectives that I find I just cannot I don't want to operate without talking to Simon in a number of topics. And one is Australia, because Australia is pushing beyond what I would have thought as the reasonable application of the Jenkins rule, which is a rule of thumb, or maybe a heuristic a word we haven't used here today. I think a heuristic is what I like, I don't care as much for the modeling. If I'm looking at modeling, I typically am going through a checklist of these rules of thumb or heuristics in my head, including the Jenkins rule to see if I feel that it's giving off good vibes or not, this is what I do before I take the huge effort that I really ought to be paid for to actually go into the details of the documentation or, or worse the, the implement implementation structure, the software implementation structure, some of these models that's hard to do. I prefer heuristics, but in Australia, Simon worked hard understanding and then being involved with the AMO, the Australian electricity market operators, I sp, the Australian ISP, integrated system plan is the leading modeling effort in the world in terms of sophistication, cost, documentation, and probably also importance to its nation's grid future. And Australia is an outstanding place in many ways for wind and solar because it's immense continent with not that many people on it, it's not so far south that it doesn't have really good sun. It's got desert areas with very little cloud cover. It's got some trickiness, and that the grids are tend to be strung out and along the coastline following the people, meaning there's some difficulty spreading out, shall we say, and taking advantage of more geography for the wind and solar. So that's a little trickier. And some of the grids are a little bit disconnected from each other compared to what you would want if you're spreading around renewables. All in all, it makes a fascinating case study almost like Texas. So Australia has spent, I think, 10s of 1000s of man hours, millions of dollars, not more than 10s of 1000s. I'll check the numbers. 10s of millions of dollars are going into this ISP plan between what's already been spent what's already what's going to be committed. It's updated every few years. It's an ongoing process since 2010, involves researchers at leading Australian universities. And although we can quibble with some of the assumptions like the way they priced in nuclear using, say the one off badly, Nick just disastrously run projects in the West as their measuring stake rather than the fleet builds that are going very nicely in the rest of the world. We can quibble about that, that I sp He seems to indicate at the moment that you can get to 99% carbonized, with sun, wind, water batteries, and some other storage technologies still to come at a reasonable wholesale electricity price. So we'll see. But there are 1000s of pages of documentation going into extreme detail about all their assumptions 1000s of assumptions about how the model works, they are an open book, they are not a black box, if it's a black box is because we're too lazy or too dumb to look into it. Not saying it will all work out, just saying that I have had to be more humble than I think a lot of people are towards these efforts. And since we are quite certainly not going to get nuclear in Australia within say, eight years, 10 years, I don't know, there's a little bit of an interesting, interesting one. And I'll never stop working with folks to pitch the insurance idea of decriminalizing allowing legalizing nuclear in Australia, I have to take the ISP effort seriously, because it takes itself deeply seriously. And I'll be following along as we see the the collision between the most sophisticated well funded modeling effort in the world and reality. And it will be interesting to see where the fault lines are, what are the problems that arise that can be fixed? If there are problems that arise? That can't be fixed? I have my doubts, call it COVID Not so much engineers pessimism, but I'm going to be a little self effacing here and say it's engineers paranoia. I have some engineers paranoia about these efforts, some of which has come out in the anecdotes I've given you but others that I don't have just I don't I don't have theoretical justification for necessarily, but that creep on me and make me uneasy with these. You call it the godlike models and sometimes the godlike modelers.

Speaker 1  56:56  

I guess, just in closing, we've talked about motivated reasoning motivated modeling might be a good catchy episode title. Thinking about the climate models from the IPCC, there's a huge sort of demand to to avoid catastrophe and Apocalypse by making models work. Whether that is you know, in the minds of researchers like Mark Z Jacobson, proving that we can, you know, get to, you know, his his exact distribution of resources and X amount of time. You know, I don't want to get into climate modeling with you here. But just I think thematically you know, I'm, I'm not a believer that net zero, is, is a possibility and a decade old timeframe, probably not a centuries from timeframe, potentially not at all. But, you know, the models need to need to fit the imperative of getting to net zero. And I think that leads to a certain amount of shenanigans and a certain amount of unserious pneus. I'm thinking about, you know, our own electricity, independent electricity systems operated in Ontario here who's come up with a net zero by 2050 plan. And it involves eating gigawatts of 16 gigawatts of hydrogen, basically, supplanting the role of gas in our grid. And I think about, you know, energy modelers in the 70s, perhaps, without the imperative of climate change, you know, guessing at what the future might look like, what our you know, resources might be, and that it would be laughable to, you know, be essentially making a, you know, a quarter of the grid, based upon, you know, an energy carrier that's very expensive to produce, the current technology is just not there in terms of, you know, generating sufficient quantities of economically like, it would be, I think, laughable, but these things are given a pass in the current moment. So just maybe, if you have any reflections on the kind of psychology of modeling in terms of the need to sort of make, make things fit with the imperative of of climate change and fears of climate catastrophe.

Mark Nelson  58:52  

Well, one of the greatest quotes on modeling is all models are wrong, some are useful, we just always must ask useful. For whom? To what end? Who benefits what's being claimed? And does that line up with what the models actually say? Or indicate even before checking to see whether the models are valid or perturbing them ourselves if we get enough funding to do so. So on the psychology of modeling, I can say that the goal shines brightly for a modeler like it may for those of us who want to build big things in the world. Both of us have that goal standing in front of us, we just want to reach out and get it for a modeler. You might get it with a better solver, and a better PR team. For us. We can only get it if we actually construct things that work in the real world and that that that is a significant difference. The Psychology of modeler, I don't think is that, that of a risk taker. One anecdote I think I would end with is that when Diablo Canyon was hanging between life and death before this right through where the governor publicly supported it and you know, crushed opposition and got Diablo Canyon widely supported by Democrats and Republicans in the House and Senate of California and, you know, Diablo Canyon looks set to be saved. Before that happened. I reached out with folks to one the possibly the leading energy modeling firm in the US and reached out to their founder their their, their, their, their head honcho, and directly said, we can get the money, the money you need, what is it 75,000 100,000 We can get that money. And we want a model comparing California without Diablo, California with Diablo on a missions and cost. And they said absolutely just any other nuclear plant other than Diablo. And I said well hold up. We don't we don't need models for other nuclear plants, other nuclear plants aren't set to close for non economic reasons. So we need one on Diablo. And the answer was, Well, no. And he said, Well, why? And they said, Well, we get we do a lot of work with the powers that be in California and we can't risk upsetting them. So you asked me about the psychology of modeling. We all have our needs, we all need to eat. Modeling serves an aim, that aim may or may not be aligned with the public good. And the public good itself may be a thing that is hotly debated between different parties. If there's to be modeling, I want to know the modelers, I want to sit down with the modelers, I want to see whether they have goals and ambitions that line up enough with mine that I can at least trust that they're putting in an effort to be right in a way that if put in charge, they would be willing to take real world risk and suffer for being wrong or suffer for unforeseen problems arising. Because most of the people I met do not seem to have this, like these, this excellent this leading firm, I just talked to you with some of the best, most trusted truly trusted by you like I would do modeling around. They were not willing to take risks in order to do what was right. That sort of thing scares me a little if as models rise and influence in government and policymaking in transforming engineered systems. And I just hope that we can add more risk to the modelers not that's not a threat. I'm just saying, we can say if you believe in your models, it's time to believe in them in ways where you take fall, you take downside of things not working out for decisions made even imperfect decisions made under the influence of your work. So that's what I would say. And that might change that like Mark Jacobson, radically, radically increasing the cost of his transformations. Once he sees a once he sees a big blackout in California in Texas that might change the way modelers work in ways that are instructive. And make modeling even wrong modeling. More useful.

Speaker 1  1:03:19  

Okay, Mark, this has been fascinating, a lot to unpack here. Obviously an area of weakness for me and one that I look forward to firming up. I've invited Jesse to come on the show and looking forward to to that listeners who are interested in more I would recommend checking out that podcast with Dr. Voltaire. Thank you enjoyed that as well mark as a as a way to, again, sort of Pierre behind the curtain of the Wizard of Oz. Understand the the thought process that are going on there. Mark, we're gonna have you back again, as I said, we gave that teaser for the state of the atom address for 2023 here. Looking forward to that. Thanks again for coming on the podcast. Always good to be here, Chris.