Blind versus Open Fire Model Validation: Issues, Pros & Cons
by Monideep Dey, PhD
Preface
This report by Monideep Dey summarizes key issues from research on fire model validation he conducted at the US Nuclear Regulatory Commission. The report was presented to the ISO subcommittee on fire safety engineering for consideration of the issues. Subsequently, the report was used in the development of ISO 16730-1:2015: Fire safety engineering — Procedures and requirements for verification and validation of calculation methods — Part 1: General, and referenced in the international standard. Monideep Dey served as Convenor of the working group at ISO that developed this international standard.
Subsequent to publication of ISO 16730:2015, Monideep Dey continued his efforts by participating as a US expert from ANSI in the development of ISO/IEC TR 17032:2019, Guidelines and examples of a scheme for the certification of processes which can be used for the certification of fire safety designs based on fire calculation methods (see Annex A.8). See the Fire Safety page for a complete description of the evolution of ISO standards he established for fire safety over a decade.
Abstract
Two international projects that examined the issue of blind (a priori) versus open (a posteriori) fire model validations, the Dalmarnock Round Robin Project and the International Collaborative Fire Model Project, have initiated a discussion in the international fire science community on the pros and cons of blind versus open fire model validations. These discussions are documented in this report and analyzed further to assist the fire science community specify a course of action for its development. The compilation and analysis of comments on the issue of open versus blind fire model validation show that although several concerns were raised against the adoption of blind fire model validations, the issues can be addressed in a standard. It is recommended that a standard be developed to phase in the use of blind fire model validations, along with open validations, in performance-based designs to achieve a higher degree of confidence in the predictive capability of the models. Third party validation can address the issue of the possible bias introduced in fire model validations by providing an independent assessment and determination of the model errors. This will add credibility to performance-based regulatory systems. A policy that accounts for the given technical limitations should be developed to guide the proper evolution of performance-based fire protection regulatory systems worldwide. The policy and standard can be revised as experience with blind validations is gained, and the technical limitations of current fire models are overcome.
See full report by Monideep Dey
See presentation by Monideep Dey to the ISO subcommittee on fire safety engineering
by Monideep Dey, PhD
Preface
This report by Monideep Dey summarizes key issues from research on fire model validation he conducted at the US Nuclear Regulatory Commission. The report was presented to the ISO subcommittee on fire safety engineering for consideration of the issues. Subsequently, the report was used in the development of ISO 16730-1:2015: Fire safety engineering — Procedures and requirements for verification and validation of calculation methods — Part 1: General, and referenced in the international standard. Monideep Dey served as Convenor of the working group at ISO that developed this international standard.
Subsequent to publication of ISO 16730:2015, Monideep Dey continued his efforts by participating as a US expert from ANSI in the development of ISO/IEC TR 17032:2019, Guidelines and examples of a scheme for the certification of processes which can be used for the certification of fire safety designs based on fire calculation methods (see Annex A.8). See the Fire Safety page for a complete description of the evolution of ISO standards he established for fire safety over a decade.
Abstract
Two international projects that examined the issue of blind (a priori) versus open (a posteriori) fire model validations, the Dalmarnock Round Robin Project and the International Collaborative Fire Model Project, have initiated a discussion in the international fire science community on the pros and cons of blind versus open fire model validations. These discussions are documented in this report and analyzed further to assist the fire science community specify a course of action for its development. The compilation and analysis of comments on the issue of open versus blind fire model validation show that although several concerns were raised against the adoption of blind fire model validations, the issues can be addressed in a standard. It is recommended that a standard be developed to phase in the use of blind fire model validations, along with open validations, in performance-based designs to achieve a higher degree of confidence in the predictive capability of the models. Third party validation can address the issue of the possible bias introduced in fire model validations by providing an independent assessment and determination of the model errors. This will add credibility to performance-based regulatory systems. A policy that accounts for the given technical limitations should be developed to guide the proper evolution of performance-based fire protection regulatory systems worldwide. The policy and standard can be revised as experience with blind validations is gained, and the technical limitations of current fire models are overcome.
See full report by Monideep Dey
See presentation by Monideep Dey to the ISO subcommittee on fire safety engineering