I don’t think any IV&V process could survive a sufficiently inappropriate series of emails. (I am not saying that is the case here. That is for others to decide. I would only caution that emails are often written as if they were private and casual, more to evoke than to inform, and thus easily misinterpreted.)

But I certainly agree with you that the present system used by the climate science community must be improved and, IMHO, along the lines you have been suggesting in this blog. In order to us to have a rational vote on public policy on climate change, we must have confidence in both the science and its embodiment in the climate models. Providing this confidence is an important mission for the climate science community. They seem to be letting us down by not adopting a conventional and consensus IV&V process.

Why they feel they need their own peculiar approach to building confidence I don’t know. The IPCC would certainly be an appropriate platform to define an IV&V consensus. But the IPCC4 report (e.g., Chapter 10) did not contain anything we would recognize as IV&V. I guess IV&V is not part of their culture.

Thanks for the info and very useful link. I have almost no experience with DAE systems, so I’m not familiar with this problem at all.

Maybe I’ll take a look into it as time from my day job permits.

https://computation.llnl.gov/casc/sundials/documentation/cvs_examples/node3.html

provides a numerical benchmark for verification of calculation of sensitivities of a simple DE system (Robertson kinetics example) but a validation of numerical values with analytical would also be useful.

The sensitivities of this simple DAE system

y’ = 1/5 (x – y ) (1)
0 = -p1 y^2 – p2 + x

are straight forward

d y’/dp1 = y^2/5
d y’/dp2 = 1/5
d x/dp1 = y^2 (2)
d x/dp2 = 1

and can be approximated by the ‘forward’ method in a numerical solution of (1).

The analytical solution of (1) is

y(t) = b0 + b4 (b1 + b2 exp(-b3 t)) / (b1 – b2 exp(-b3 t)) (3)

x(t) = p1 y(t)^2 + p2

a = -4 ( p1 p2 + 1 ) + 5
b5 = sqrt(a)
b0 = 1/(2 p1)
b1 = 5 + b5
b2 = 5 – b5
b3 = b5/5
b4 = -b5/(2 p1)

p1 and p2 real such that -4 ( p1 p2 +1 ) + 5 > 0, say p1 = -5.837, p2 = 2.458.

(3) satisfies (1) but the analytical sensitivities calculated from (3) have values different from (2).

However, note that Schmidt’s in-line responses contain not a single piece of data. Instead it is filled with wishy-washy assumptions and hypotheticals. In my career whenever there have been technical issues with models, methods, and calculated results you could only come to the table if you had theory and calculations in hand to support your points of view. Without these, discussions have no end and problems are not solved.

[Response: Fine. But that doesn't mean that every idea from outside has merit. Quoting Carl Sagan (almost) "They may have laughed at Galileo, but they also laughed at Bozo the Clown." Ideas and contributions have to evaluated on their merits, not from where they come from. [my bold] – gavin]

hmmm I wonder what has happened to; Peer-Reviewed Publications, Not a Certified Climatologist, High-Impact Factors, Properly Certified Climatological Journals, . . . etc, etc, etc. and the almost endless list of other stupidly applied false requirements. Not to mention the endless labeling and foul debasing characterization of persons and their contributions..

I could hardly believe what I read. Apparently neither could Jonas N in this comment.

To which gavin said:

[Response: Apparently you did. This post is about the overhyping of technical details into critiques and false assertions about scientist's integrity and the dramatic jumping to conclusions evident in the various responses. It is not, and never has been about the right of people to question or investigate the science themselves [my bold] (despite what you might read elsewhere). – gavin]

It’s clear to me the gavin does not read the comments at RealClimate

Deterministic 3D strange attractors are beautiful…but very rare !

No more ten.

Lorenz, Rossler, Chua, Gilpin, Ueta & Chen…and I !

Indeed, I built a class of Strange attractors with a system of three ODE ( only one is nonlinear).

Research intitled:

Feedback Loop in Extended van der Pol’s Equation Applied to an Economic Model of Cycles, and published in the “International Journal of Bifurcation and Chaos”, Vol. 9, N°4 (1999), pp. 745-756.
In the same volume of the Chen ODE System article.

Presentation ( pictures and graphics ) in :

http://chaos-3d.e-monsite.com

Kindest Regards,

Safieddine Bouali

Appendix A in that document outlines a method that has been successfully applied, several times, to large, complex engineering software after-the-fact. The general concepts given there are made specific for the particular models and methods and applications under review. Because the results of applying the method to software are almost always considered to be Quality related items, they are Controlled documents and not usually available for distribution to outside parties.

The documentation of the models, methods, coding, and applications are taken to be the specification documents for the software and IV&V tasks are developed from the table of contents of the documents. The specification documents are taken to, roughly, be the blueprints and the objective of the IV&V exercise is to determine that the as-built product corresponds to the blueprints. Unfortunately, many codes that have evolved over decades of time do not have sufficient documentation for the process to even get started.

Let me know if you want to pursue this further; maybe I can offer additional information for a specific case.

Thanks for the link to the ASC V&V program summary. It was very interesting. Is there any overview document like this that describes what the climate model ensemble IV&V program plan would need to look like — assuming the Climate Change Community finally decides to commit to IV&V?

I am aware of your background article: Discussions of Application of Verification, Validation, and Quality Assurance Procedures to Climate Modeling Software. Have you done anything more detailed and specific? Perhaps something that discusses SQA remediation? (It is much more difficult to IV&V a software development process after-the-fact rather than during development.) Having some expertise in the area and some free time, I was thinking about investigating the issues myself.

What a worthless pile of garbage. Their response actually demonstrates how weak their case is. They’d have been better off to ignore you.

[WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.