Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for

the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

Photos placed in horizontal position with even amount of white space

between photos and header

To replace these boxes with images open the

slide master

MELCOR

New Modeling

SQA

Utilities

MELCOR 2.1 Software Quality Assurance Program

Presented by Larry Humphriesllhumph@sandia.gov

1

MELCOR Code Development MELCOR is developed by:

US Nuclear Regulatory Commission Division of Systems Analysis and Regulatory Effectiveness

MELCOR Development is also strongly influenced by the participation of many International Partners through the US NRC Cooperative Severe Accident Research Program (CSARP) Development Contributions – New models Development Recommendations Validation

MELCOR

Code

Developmen

tNew Modeling

New/improved modeling

Code Performance

SQA

Validation

QA

Numerical Stability

Utilities

SNAP

NotePad++ library

Uncertainty Tools

What is the MELCOR Code Designed for reactor severe accident and

containment DBA simulation PWR, BWR, HTGR, PWR-SFP, BWR-SFP

Fully Integrated, engineering-level code Thermal-hydraulic response in the reactor

coolant system, reactor cavity, containment, and confinement buildings;

Core heat-up, degradation, and relocation; Core-concrete attack; Hydrogen production, transport, and

combustion; Fission product release and transport

behavior Desk-top application

Windows/Linux versions Relatively fast-running SNAP for post-processing, visualization, and

GUI

4

MELCOR Applications Forensic analysis of accidents –

Fukushima, TMI, PAKS State-of-the-art Reactor

Consequence Analysis- SOARCA License Amendments Risk informed regulation Design Certification Preliminary Analysis of new

designs Support of International

Regulatory Bodies Non-reactor applications

Leak Path Factor Analysis

5

MELCOR Code Development History

MELCOR 1.8.2 (1993) One of the earliest versions for

widespread release. Not recommended for use

MELCOR 1.8.3 (1994) BH Package CORCON-MOD3 Not recommended for use

MELCOR 1.8.4 (1997) Retention of molten metals behind oxide

shells Vessel creep rupture model Flow blockage model Radiant heat transfer between HSs Hygroscopic aerosols, chemsorption on surfaces, SPARC 90

MELCOR 1.8.5 (2000) CF arguments could be added to plotfile Consistency checks on COR/CVH volumes Iterative flow solver added Diffusion flame model SS & NS components added for structural

modeling Upward & downward convective &

radiative heat transfer from plates Particulate debris in bypass introduced Improvements to candling, debris

slumping, and conductive, radiative, and candling heat transfer

PAR model was added CsI added as a default class Improvements to hygroscopic model Iodine pool modeling Carbon steel was added to MP package

6

MELCOR Code Development History MELCOR 1.8.6 (2005)

An option was added to generate input for the MACCS consequences model.

Input was added to simplify conformance with the latest best practices (now defaults in 2.x)

New control functions (LM-CREEP & PIP-STR) for modeling pipe rupture

Modeling of the lower plenum was revised to account for curvature of the lower head

Formation and convection of stratified molten pools

Core periphery model for PWRs to model core baffle/formers and the bypass region

Reflood quench model Oxidation of B4C poison Release of AgInCd control poison Column support structures was added Interacting materials added to allow modifying

enthalpy tables Spent Fuel Pool modeling Flashing model Modified CORSOR Booth release model added Jet impaction model Hydrogen chemistry models

MELCOR 2.x (Beta release in 2006) Code internal structure greatly modified Dynamic memory allocation New input format Formula type control functions New HTGR modeling (PBR, PMR) Counter-current flow model Point kinetics model Smart restart Simplified accumulator model Ability to track radionuclide activities Turbulent deposition model & bend

impaction Control function for deposition mass for

each deposition mechanism. MELCOR/SNAP interaction in real-time Full report to user of sensitivity values Cell-based porosity Spent fuel pool models Intermediate heat exchanger /machinery

models Hydrogen chemistry models

7

MELCOR Users Worldwide

Argentina Belgium Bulgaria Canada China Czech Republic Finland France Germany India Italy Japan Mexico Netherlands Russia S.Korea Slovak Republic South Africa Spain Sweden Switzerland Turkey UAE UK USA

DOE Safety Software Software Quality Assurance

A self-assessment was performed on September 6, 2007 with the following objectives: Provide MELCOR software team and management with a perspective of

areas of software engineering/quality strength and areas of improvement - relative to the self-assessment instrument and requirements of DOE O 414.1C.

Provide a comparison of the assessment results and gaps for MELCOR with the gap analyses conducted by DOE in 2004.

Provide MELCOR with a potentially sustaining capability to conduct their own self-assessments and manage identified improvement areas.

Sandia Corporate Process Requirement 001.3.6 (CPR 001.3.6) The software management framework adapted from two

internationally recognized standards the Capability Maturity Model Integration (CMMI) ® and ISO 9001 These standards provide elements of traceability, repeatability, visibility,

accountability, roles and responsibilities, and objective evaluation

9

Software Quality AssuranceAnnual Re-evaluation

• Process areas– Project planning and oversight, PPO– Risk Management, RSK– Requirements Development and Management,

RDM– Technical Solution, TS– Verification and Validation, VV– Development and Lifecycle Support, DLS– Configuration Management, CM– Measurement and Analysis, MSA– Integrated Product, IPD– Integrated Teaming, ITM

– Process Dimensions– Stakeholder Involvement , SI– Ongoing Process Monitoring and Control, PMC– Collected Improvement Data, CD– Objective Evaluations, OEV– Quantitative Objectives Defined for Processes, QPO – Stable Subprocess Performance, SSP – Training , TR– Problem Reporting & Corrective Action, RCA

10

MELCOR Software Quality Assurance Best Practices

MELCOR Wiki Archiving information Sharing resources (policies, conventions,

information, progress) among the development team.

Code Configuration Management (CM) ‘Subversion’ TortoiseSVN VisualSVN integrates with Visual Studio

(IDE) Code Review

Code Collaborator Nightly builds & testing

DEF application used to launch multiple jobs and collect results

HTML report Regression test report

Regression testing and reporting More thorough testing for code release Target bug fixes and new models for

testing Bug tracking and reporting

Bugzilla online Validation and Assessment calculations Documentation

Available on Subversion repository with links from wiki

Latest PDF with bookmarks automatically generated from word documents under Subversion control

Links on MELCOR wiki Sharing of information with users

External web page MELCOR workshops Possible user wiki

Emphasis is on AutomationAffordable solutionConsistent solution

11

MELCOR Developers Wiki Site – Internal Use

MELCOR Developers Wiki Archive records

Requirements, design, and testing

Regression tests Assessment work

Information Sharing Debugging Policies Testing Policies Code Development

practices Coding Conventions Lessons Learned Software Risk

Management Version Changes Reference Library

12

Software Configuration Management (SCM) Methodology for managing code changes in a team

development environment All SCM systems provide the following essential features:

Concurrency Management Concurrency refers to the simultaneous editing of a file by more than one

person. – Resolve places where code changes conflict

Versioning Tracks file versions Makes it possible to roll back changes or recreate a version

Synchronization Update changes made by other developers

13

Subversion (SVN) Free Open Source Operates across networks allowing

different people access to a central repository

Atomic commits. No part of a commit takes effect until

the entire commit has succeeded. Revision numbers are per-commit, Commit's log message is attached to its

revision Branching

Merging branches back into the trunk Repository can be locked

Especially useful for binary files Command Line Client

Makes it possible to integrate SVN with other applications

TortoisSVN Subversion client that integrates SVN

with Windows Explorer

14

Full Compiler/Linker Options Encoded in MELCOR Compiler/Linker options printed in output and diagnostic file

Provides a QA check for reproducing results

Is also accessible by running executable with command-line arguments

Melcor.exe -config

Configuration information for MELGEN Subversion revision: 1951

Compiler: IVF 11.1\065

Compiler settings: /nologo /O3 /fpp /Qparallel /DLICENSING_INACTIVE /DWINDOWS /real_size:64 /Qaut o /align:rec8byte /Qtrapuv /module:'E:\CustomBuild\MELCOR_2x\r1951\O2_CVF_IBRAE \modules' /traceback /fp:source /Qprec-div /Qprec-sqrt /assume:protect-parens / assume:buffered_io /libs:static /threads /dbglibs /c /assume:nocc_omp /QaxSSE4 _2,SSE4_1,SSSE3,SSE3,SSE2,SSE

15

Code Review SQA Requirements:

As outlined in both Sandia Corporate Procedure:IM100.3.2 and ASME NQA-1, an important aspect of a Software Quality Assurance program involves review and documentation for the entire life cycle of software development, from requirements and design to implementation and testing.

Benefits Code reviews can reduce number of defects in new code Code reviews can lead to better documentation of code & better

understanding of new modeling among team members Code reviews provide a process by which seasoned programmers can pass

experience and knowledge to less experienced programmers. Improves code readability

Problems Code reviews can be time consuming or cursory Code reviews are not well documented Reviews can be adversarial and not productive

16

CodeCollaborator• HTML based collaboration

– Browser is all that is needed to access– Shared licenses for multiple users

• Threaded, contextual chat– File changes, chat conversations, and defects

are linked together. – Each conversation is threaded by file and line

number and can be viewed simultaneously with file content.

– Choose between multiple views, or download differences to your local machine for further inspection and testing.

– Hyperlink directly to a file or line numbers

• Version Control Integration– Integrates with Subversion

• Asynchronous Review– Perform and manage reviews even when

participants are separated by many timezones. – Comments are tracked like newsgroups

– Review PDFs or Image files

17

MELCOR Testing Overview

Unit Testing All input options should be tested Ranges of input should be tested.

Values outside of reasonable input should be tested for error messages Automated Build & Test

Test all revisions and ‘catch’ revisions that break the build Build on multiple platforms (currently only Windows) and compilers (CVF & Intel) Suite of fast-running test problems Objective is to correct problems sooner MOE utility for searching output for specified test for success criterion

Code Release Testing Larger suite of test cases Test recent bug fixes Test new modeling

Code Test Coverage Code Profiling

18

MELCOR Code Testing Build Testing

Automated to perform Nightly Builds MELCOR 1.8.6 Windows Compaq Visual

FORTRAN (CVF) MELCOR 2.X Windows CVF MELCOR 2.X Windows Intel Visual

FORTRAN (IVF) MELCOR 1.8.6 Linux IVF MELCOR 2.X Linux IVF

Using CMAKE to generate make files for use on Unix variants to extend building on other platforms & other compilers

Code Testing Performed Daily (at least frequently) Standard test cases chosen for physics

coverage over 140 test cases New cases added regularly Debug & optimized versions tested Testing of developmental branches

Comparison of results Consistent results between Windows &

Unix Consistent results between SNL & IBRAE

19

Optimization Studies Optimization can lead to different results

Uninitialized Variables (this is just one possible reason) Can make it difficult to reproduce errors observed in the release version in

the debugger Optimization Utility to test optimization of each file

Optimized and unoptimized versions of all object files are built An entire module is tested for optimization first (optimized library is linked

with debug library for all other modules) A fast running test deck is run and the output is compared with the debug

version for differences If differences are observed, each file in the module is tested individually.

Testing of optimized files is performed in parallel Results indicate that only a few files cannot yet be optimized. Optimized executables are built with these files unoptimized

Give identical results to debug version !!!

MELZILLA Bug Reporting Bugzilla site for bug reporting, tracking, and

information Available from SNL web page

Users submit bugs and details OS, Hardware, affected packages, severity Bug description Attachments

Comments and attachments can be marked private and not visible to other users

20

SVN/Bugzilla/Visual Studios/Testing Integration

Integrates Subversion with Windows explorer All commands are available directly from the windows explorer. See the status of files directly in the Windows explorer Allows moving files by right-dragging them in the windows explorer

Integration with issue tracking systems A separate input box to enter the issue number assigned to the commit,

or coloring of the issue number directly in the log message itself When showing all log messages, an extra column is added with the

issue number. You can immediately see to which issue the commit belongs to.

Issue numbers are converted into links which open the web browser directly on the corresponding issue

Optional warning if a commit isn't assigned to an issue number Integration with MSWord compare Integration with Visual Studio

All modifications apparent within Visual Studios Easy to see and check in all modified files (still not automatic)

21

22

MELCOR: Self-Documenting Code

MELCOR generates a complete list of MELCOR Keywords Global record ‘PrintInputRecords

<filename>’ Part of required input processing

routine means that all records recognized by MELCOR are printed

MELCOR generates a list of control function arguments recognized by MELCOR Enabled by ‘PrintInputRecords’

MSWord Macro that scans the user guide document for input records and CF arguments Comparison with MELCOR list

enables identification of undocumented keywords

23

MELCOR Quality Assurance: Tracking Code Changes Changelist

List of code issues and modifications by revision

References to bugzilla site MELCOR Trends

Provide a very general assessment of code modifications

Code stability Performance Metrics

– H2 generated, Cs deposition, deposition on filters, CAV ablation

Provided with each public code release

Automated as part of testing

24

MELCOR Code Validation Both Separate Effects and Integral Tests Part of our regression test suite

Many of these are ongoing analyses

Participation in multiple International Standard Problems

Coverage of most important physics Heatup/Heat transfer Oxidation Reflood Degradation Molten pool FP Release Vessel failure Critical Flow MCCI DCH Condensation Containment stratification Hydrogen Burn Hygroscopic effects Aerosol deposition RN transport Iodine pool chemistry Suppression pool level response Vent clearing Engineering Safety Features

Sprays Ice Condensers

Sensitivity Analysis of Input Analysis Will be published as Volume III Documentation

Report

RN Transport• FALCON 1 & 2• VANAM-M3• LACE-LA4• LACE-LA1 & LA3• STORM• AHMED• ABCOVE• CSE-A9• DEMONA• RTF ISP-41• VERCORS• ORNL VI• MARVIKEN ATT-4

Containment• NUPEC M-8-1, M-8-2• IET 1 through IET7 and

IET 9 through IET 11• PNL Ice condenser

tests• Wisconsin flat plate• DEHBI• CVTR• HDR V44• HDR E-11• NTS-Hydrogen Burn• GE Mark-III

Suppression Pool• Marviken Blowdown

Tests• CSTF Ice Condenser

test• LOFT-FP2

COR heatup,degradation, & FP release• LOFT-FP2• PBF-SFD• CORA-13, Quench 11• DF-4, MP1, MP2• FPT1, FPT3• LHF/OLHF• VERCORS• ORNL VI

Ex-Vessel• OECD-

MCCI• SURC• IET-DCH

Integral Tests/Accidents• Bethsy• Flecht-Seaset• GE Level Swell• RAS MEI• NEPTUN• TMI-2• FUKUSHIMA

Assessment Process

After phase III• Documents go to editor

for final review• Review & Approval

25

Phase II(ongoing)

Update/ initiate assessment

Update spreadsheet

Presentation at weekly assessment meeting

Commit all files (decks, XLS, Word) to repository

Transfer test case to another analyst for Phase III reviewAssessment should be complete

Phase III(ongoing)

Second review (presentation if necessary)

Re-run with final code version

Clean up input deck

Draft of write-up

26

MELCOR User Workshops and Meetings MELCOR Workshop

September 12-15, 2011 More than 70 participants Use of SNAP

September 10, 2012 MELCOR Code Assessment

Program (MCAP) September 22-23, 2011 September 13-14, 2012

European MELCOR User Group (EMUG) ENEA: April 11-12, 2011 GRS : April 16-17, 2012

EMUG 1

EMUG 2

EMUG 3

EMUG 4

27

Questions?