Models, Methods, Software

I received the following e-mail from Pat Roache regarding his new book about Verification and Validation. He has also made arrangements for reduced prices on two others of his books.

I am pleased to announce the publication of
“Fundamentals of Verification and Validation” by Patrick J. Roache.
Copyright 2009, ISBN 978-0-93478-12-7. 476 pages, subject index.

The book is the successor to my 1998 book “Verification and Validation in Computational Science and Engineering.” About 1/3 of the material is new, including a new Chapter 11 describing the Total Validation Uncertainty approach of ASME ANSI Standard V&V 20 (2009).

The attached file outlines the new features of the book. A complete Table of Contents, including designators for new and modified Sections, will be found on the website.

The price is the same as the 1998 book, U.S. $85.00 (but shipping charges have increased). The book is available directly from our fulfillment house, BookMasters, by email order to

orders@BookMasters.com

or from Amazon.com at the following.
http://www.amazon.com/Fundamentals-Verification-Validation-Patrick-Roache/dp/0913478121/ref=sr_1_7?ie=UTF8&s=books&qid=1253150836&sr=1-7

The orders to BookMasters tend to ship faster since they do not run out of stock. Also, wholesale orders can be placed by contacting BookMasters at orders@BookMasters.com.

I would appreciate it if you would forward this email to any of your colleagues who might be interested.

Thanks for your consideration.

Respectfully,
Patrick Roache

p.s.
If anyone is interested, the 1998 V&V book is on clearance sale for 1/2 price at US $42.50. Available from orders@BookMasters.com or from Amazon at
http://www.amazon.com/Verification-Validation-Computational-Science-Engineering/dp/0913478083/ref=sr_1_4?ie=UTF8&s=books&qid=1253718713&sr=8-4.

Likewise, the book “Fundamentals of Computational Fluid Dynamics” is on clearance sale for 1/2 price at US $37.50. Available from orders@BookMasters.com or from Amazon at
http://www.amazon.com/Fundamentals-Computational-Dynamics-Patrick-Roache/dp/0913478091/ref=sr_1_5?ie=UTF8&s=books&qid=1253718713&sr=8-5

[Updated September 26]
I think the name of the program originally known as the Advanced Strategic Computing Initiative ( ASCI ) is now known as Advanced Simulation & Computing ( ASC ).

[Updated August 26]

I have uploaded an excellent summary, developed by Los Alamos, of the ASC V&V Program at LANL. It is here.

A couple of quotes.

Confidence in simulation extrapolation comes via confidence in physics & numerics models, not calibration to experimental data.

Having “good agreement” between calculations and observations is not sufficient to establish scientifically credible predictive capability.

More and more it seems that the Climate Change Community remains the only holdout among all compute-intensive enterprises relative to application of rigorous, independent V&V to computer software.

Update
Read the rest of this entry »

I sent a slightly revised copy of the comment on the Proposed CO2 Ruling, given in this post, to Ms. Lisa jackson, Administrator of EPA. The letter to Ms. jackson is shown below. I received a reply from Rona Birnbaum, Chief, Climate Science & Impacts Branch, Climate Change Division. (The actual signing of the letter was a task delegated to someone whose name I can’t read.)

As in all previous cases in which I have attempted to convey the critical necessity of Independent Verification and Validation to persons outside the software development community, I failed again. It is obvious that Ms. jackson, Rona Birnbaum, or whoever read and responded to the letter, have no idea what I’m talking about.

Read the rest of this entry »

I have started working on a toy model and plan to include analytical sensitivity analysis as part of the methods.  These notes, and an associated extended discussion that I have up-loaded, serve as a short introduction to the subject.

The file is here:

Summary

These notes introduce a few of the ideas and concepts associated with sensitivity analysis for algebraic and ordinary differential equations. By sensitivity I mean what are the effects of changes in the numerical values of the parameters in a system of equations relative to a response function of interest. The response function can take any mathematical form, but I will focus on the values of the dependent variables of the equation system.

Read the rest of this entry »

The focus on this previous post was the fact that approximations made at the continuous-equation level mean that the model’s mass and energy budgets are different from the mass and energy budgets of the physical system. Note that there are very significant additional issues associated with the discrete approximations applied to the continuous equations and the numerical solutions of these. These issues in the discrete domain, in my opinion, have the potential to far outweigh issues in the continuous domain. Accurate integration of the discrete approximations over the enormous times scales of interest is a very tough problem.

The present post looks at some more issues associated with energy conservation in the discrete domain.

Read the rest of this entry »

Some time ago it was suggested to me something to the effect that most of the people in the Climate Change Community to whom I suggest V&V and SQA issues are critical and require significant attention don’t even know what I’m talking about. Here’s some information that lends support to that observation.

Read the rest of this entry »

The Intergovernmental Panel for Climate Change (IPCC) Fourth Assessment Report (AR4, IPCC, 2007) states:

“A major advance of this assessment of climate change projections compared with the TAR is the large number of simulations available from a broader range of models. Taken together with additional information from observations, these provide a quantitative basis for estimating likelihoods for many aspects of future climate change.” [ My bolding. ]

Do the numbers from these “large number of simulations available from a broader range of models” GCM calculations have any Meaning. My answer is No.

One crucial and necessary first step is that application of Verification procedures have shown that the numbers produced by the software accurately reflect both (1) the original intent of the continuous equations for the models, and (2) the numerical solution methods applied to the discrete approximations to the continuous equations. That is, Verification shows that the equations have been solved right. Do the numbers actually satisfy the Verified-to-be-correct-as-coded discrete equations and do the solutions of the discrete equations converge to solution of the continuous equations. Neither of these has been demonstrated for any GCM. I will be pleased to be shown to be wrong on this point.

All software can be Verified. Objective technical criteria and associated success metrics can be developed and applied in a manner that provides assurances about the correctness of the coding of the equations and their numerical solutions. Lack of Verification leaves open the potential that the numbers from the software are simply results of “bugs” in the coding.

The present-day software development community, in all kinds of applications and organizations, is keenly aware that lack of SQA policies and procedures, and successful applications of these to the software, leaves open a significant potential for problems to exist in the software. So far as I am aware, there are no precedents whatsoever for public policy decisions to be based on software for which no SQA procedures have been applied.

The journal of Computational Physics has a special issue that might be of interest to many here, Predicting weather, climate and extreme events.

This is a review article.

And Elsevier, aka Big Science Publishing, has kindly provided links to 8244 related articles.

Introduction
The calculations preformed with the equation systems are summarized in the following discussions. The focus had been on testing for convergence of the numerical solution methods to solutions of the continuous equations. By convergence I mean that as the size of the discrete increment for the independent variable is reduced the calculated values for all dependent variables approach limiting constant values for all values of the independent variable.

None of the systems that are said to exhibit chaotic response have shown convergence. One of those, the Terman system, exhibits periodic response, not chaotic response. The Saltzman system was never intended to be an example for chaotic response.

Read the rest of this entry »

The numerical solution methods that will be used to check convergence are given in a file that I uploaded.

Let me know if you see any typos or if you want to see some results for a specific equation system.

I’m thinking that Part 1d will be some numerical results.